This paper is a short summary of points that are important for us, from the large literature on CSCW - which stands for Computer Supported Cooperative Work, a subfield of HCI that is particularly concerned with the social aspects of interface design. This paper is very condensed, and uses some terminology that may not be familiar. For example, the word "nuanced" refers to the very common tendency of people to match what they say and do to the particular situation that they are in; e.g., most people speak differently to babies, todlers, elementary school children, and native speaking adults; they also behave differently at a tuxedo-wearing formal dinner party than at a beach party. In fact, almost anything that humans do exemplifies nuance. (The phrase "recipient design" is used for a similar but more precise concept in the field of ethnomethodology, as discussed in more detail below.)

Many large systems - and even some companies - have failed through ignorance of points that Ackerman makes. For example, the UK National Health Service sponsored a number of multi-million pound information systems for hospitals that failed because the doctors and nurses who had to enter the data disagreed strongly with the philosophy of health care that was built into these systems, and therefore sabotaged them by not using them and/or by entering misleading information. "The Coordinator," an ambitious system intended to improve corporate communications, failed and brought down the company that built it, because it tried to impose a communication style on people that it turned out many of them hated, because it was impossible to frame many communications in an appropriately nuanced way. (This system is discussed briefly on page 484 of the third edition of Shneiderman.) There are many many more examples like these.

Semiotics is the study of signs. Signs mediate meaning, and are not just simple "tokens" or physical marks; they can be complex combinations of other lower level signs, such as whole sentences, spoken or written, newspaper advertisements, whole books, etc. The American logician Charles Sanders Peirce (pronounced "purse") introduced the term "semiotics," and several of its basic ideas. In particular, he emphasized that meanings are not directly attached to words; instead, there are events (or processes or activities) of semiosis - i.e. constructions of meaning - each involving a signifier (called the representamen by Peirce), a signified (an object of some kind - e.g. a concept), and an interpretant that creates the link between these two; these three things are often called the semiotic triad and occur wherever there is meaning. The interpretant is not given, but must be created, e.g. by a person. This sounds simple, but it is very different from more common and naive theories, as in the use of denotational semantics for programming languages, where we have a function from programs (which are signs) to their denotations (which are meanings), in general defined by higher order functions on some rather abstract mathematical domains, or more simply, by a compiler to a particular machine.

The second main source of semiotics is the Swiss linguist Ferdinand de Saussure. In both traditions, signs can be anything that mediates meaning, including words, images, sounds, gestures, and objects. In the tradition of Saussure, every sign has:

• a signifier, which is an associated form; and
• a signified, which is a concept that it represents.

Signifier: The letters 't-r-e-e'.
Signified: The concept of tree.

Peirce's definition of sign is better, though more complex, because it includes the creation of the link between signifier and signified as an explict component; this can include the interpreter, the context of interpretation, and the process of interpretation. Peirce's notion is triadic, whereas Saussure's is only dyadic (i.e., binary). Although Peirce's notion, like Saussure's, is relational, in contrast to Saussure, it is open rather than determined in advance, as well as triadic. In our application area of user interface design, the interpreter is usually course the user or the designer. More detailed discussion appears in On Notation.

Yet another important notion from semiotics is Peirce's three way classification of signs into symbols, icons, and indices. These concepts have many applications to user interface design; again see On Notation for details. You should carefully note that each of these three terms has a technical meaning that is not the same as its ordinary everyday meaning!

Saussure's most important insights are probably that signs come in systems, not just one by one, and that they can have internal structure. Another insight of Saussure that is often not sufficiently emphasized, is that sign systems are organized by systematic differences among signs; we can relate this to a famous saying of Gregory Bateson, that "information is a difference that makes a difference." Saussure's idea that signs come in systems is illustrated by examples like the vowel systems of various accents of the same language, and the tense systems for verbs in various languages. The vowel system example shows that the same sign system can be realized in different ways; we call these different models. The vowel system example also shows that two different models of the same sign system can have the same elements but use them in a different way; so it's how elements are used that makes the models different, not the elements themselves. Models of sign systems are not just sets, they are sets with some kind of structure; we will learn more about this later. Alphabets also provide good examples where the sets overlap; for example, the Greek, Roman and Cyrillic alphabets each have some tokens in common; this motivates the need for "signs" as tokens that come in systems and have an interpretation; they cannot be just tokens as such. We can also motivate the need for systems of signs by noting that a sign system with just one element cannot convey any information (more technically, this is because its Shannon information content is zero).

Algebraic semiotics attempts to combine the major insights of Peirce and Saussure (among others) into a precise formalism that can be applied to the practical engineering of sign systems, e.g., in user interface design. This involves a fundamental change in perspective, from the analytic perspective of traditional semiotics, to a synthetic perspective, which is concerned with construction rather than just analysis. One important insight that this formalism incorporates is that signs need not be the simple little things that we usually call "signs," but instead can be very complex, such as a book, or a series of books, or even a whole library; or a movie or series of movies, or the GUI to an operating system. It is the job of user interface designers to build (parts of) such systems. Another insight that algebraic semiotics pursues is the importance of studying errors, that is, badly designed sign systems, in order to better understand what it means for a sign system to be well designed. This is interest is illustrated in the exhibits of the UC San Diego Semiotic Zoo.

Later we will see that it is useful to view sign systems as abstract data types, because the same information can be represented in a variety of different ways; for example, dates, times, and sports scores, each have multiple representations. This leads naturally to the idea that representations are mappings between abstract data types, as illustrated in an informal way by the examples in the UCSD Semiotic Zoo, which show how the failure of a representation to properly preserve some structure results in its being a suboptimal representation.

Traditional semiotics seems to have a Platonistic bias, in assuming that signs (which, please recall, we take to include token, meaning and their relation) have an existence which is independent of human beings. But here we take a quite different approach, in basing signs on human social activity. We claim that signs do not have meanings in themselves, but only have whatever meaning we give to them. Moreover, insofar as these meanings are shared, they are necessarily social, and Plato's ideal entities play no role in this.

Later in this course we will go much deeper into the structure of signs, and the representation and interpretation of signs. Peirce and Saussure are only the beginning of the story.

An important distinction in considering methods to support interface design is that between qualitative and quantitative methods. The latter have a long history in pure science, e.g., physics, whereas the former have an even longer history, being part of what it means to be human, though the name of course is recent. In the early days of CHI, many hoped that quantitative methods would be sufficient, and ergonomics and experimental psychology ruled; for better or worse, this has failed, and now qualitative methods rule, though there are many who still wish for a more "scientific" approach, and a brave few who try to make it happen. The problem is that "quantitative" means having numerical measures, which in general requires having some specific fixed situation in which measurements can be taken; but this is often not possible for the issues that are of greatest interest, since the system does not yet exist, or if it does, the designer is there precisely because it needs to be changed! Moreover, numerical measures are often unavailable or inappropriate for many social phenomena of interest.

Another distinction is more philosophical, between construcitivists and realists. Constructivists believe that the way we see the world is in large part built by processes of perception and understanding (or semiosis!), whereas realists believe that on the whole, we perceive the world the way it really is. Interface designers today are more likely to be constructivists than realists, because they know from experience that users can construe interfaces in very many different ways, and that their job is to get users to construct a certain a small part of the world in a certain particular way.

In particular, direct manipulation is the illusion that one is directly manipulating something real, such downloading a file from a remote machine to a local directory by picking it up and moving it over. As an exercise, it is interesting to think about all the different ways in which this really is an illusion. Nonetheless, one might say that the illusion is real enough, if the user does not think about the interface at all, but only about the task, and describes that task using a metaphor of directly manipulating objects. In fact, it is an important general principle of interface design that the interface is successful to the extent that it is invisible; if the user must pay attention to the inteface instead of the task, then something is wrong. This principle is sometimes called transparency. Affordances and visualizations are other ways (other than direct manipulation) that transparency can be achieved.

Change is inevitable, unending, and unpredictable, especially in user interface design. Many changes are related to (what has been called) convergence, which is the merging of computers with communications, plus of course "Moore's Law" (which is not a law at all). In user interface design, this can be seen in the evolution of HCI towards CSCW: whereas traditional HCI focuses on an individual user of a single system, CSCW addresses the social issues that necessarily occur when individuals communicate within communities, which of course they (nearly) always do. The effects of change are illustrated by the webnote Art and the Zen of Web Sites, much of which is now badly out of date.

One aspect of the original HTML philosophy (as developed at CERN, a physics research center in Switzerland) is that source texts should only use structuring commands and never use layout commands, so that browsers are able to choose the best presentation compatible with their capabilities, e.g., audio output for blind users. Now consider the HTML source code for the following "button sign" constructed from a table with background:

Is this complex sign consistent with the original HTML philosophy described above? (The best way to learn more about HTML is to use your browser to look at source code.) Note that the HTML language has changed a great deal, in response to pressure from users, such as advertisers, with very different goals than the original community of high energy physicists.

Recipient design is a term from ethnomethodology that refers to the phenomenon that natural speech is always designed for its specific recipients in a specific social context, and usually bears specific evidence of that design. It is a good exercise to find examples of recipient design in natural language used in everyday life; almost anything can be seen in this light. Recipient design is as defined above is more precise than nuance, which refers to the contextualizations that people use to make signs (e.g., speech) work in everyday life; note also that nuance carries the connotation of being subtle. A brief webpage on speech act theory and mitigation has been written for this course; mitigation is a very interesting example of receipient design, and it is precisely what "The Coordinator" did not allow.