A very fundamental question that any course on user interface design ought to address is
"What is design?"One way to approach this question is to examine the linguistic structure of the word "design." There are two morphemes, "de" and "sign", so that "design" literally refers to a thing that is derived from signs; it is interesting to notice that "design" shares its first two morphemes with the word "designate" ("de" + "sign" + "ate"). Through the sort of semantic evolution to which societies commonly subject their words, "design" as a verb has come to mean creating some configuration of signs, for some particular purpose. So user interface design is about creating configurations of signs, today usually on a computer screen, and also usually for some practical purpose.
Even a quick look at the HTML language can give a sense of what are today considered (for user interface design) the most basic kinds of sign, and the most important ways to derive complex signs from other signs: letters and images are the most basic kinds of sign; and paragraphs, lists, tables, etc. are the most important devices for derivation. The result of designing a webpage in HTML is a complex sign, derived in a carefully described way from a number of specified parts. The same can be seen in other languages that support interface construction, though perhaps less clearly.
Thus, our subject is in a very fundamental way involved with signs, and in particular, with how to construct complex signs out of basic signs, in order to effectively accomplish some practical aim. There is much in common with design in other media, such as architecture, magazine layout, and watercolor painting, including artistic dimensions such as good taste, creativity, flexibility, insight, etc., but there are also some significant differences; in particular, the structuring devices and the goals are usually much more explicit, and much more limited. This is an ideal situation for applying semiotics, and because semiotics is at an early stage of development, it is also an ideal situation for supporting its further development. (See the discussion in the very short paper One Cannot Not Interact by Mihai Nadin.)
Another very fundamental issue in user interface design is how knowledge is validated, that is, how can we find answers to the questions that are raised? These questions naturally divide into two groups: (1) basic scientific laws and engineering principles; and (2) properties of particular designs. The study of such problems is called methodology, though in computer science, this work is more often used as a synonym for "method". A traditional view of user interface design says that the scientific methods of experimental psychology should be used for both of these. While experimental psychology may have been a good place from which to start the study of interface design, over time, its limitations have become more and more evident. For example, the following rather naive description of "The reductionist scientific method" appears on page 28 of Shneiderman's widely used text (third edition):
Materials and methods must be tested by pilot experiments, and results must be validated by replication in variant situations.
Many of the complications that make it difficult to apply the scientific method arise from the fact that the actual use of an interface is always situated in some social situation. A simple example of this is how the physical distribution of workstations can affect the ease of accessing knowledge about software packages. If all workstations are located in some common space, then it is easy for users to share information, but much more difficult if each workstation is in a separate cubicle or office. Some often used alternatives to scientific experimentation include stylistic guidelines, and usability testing, both of which are much less expensive and time consuming.
A rare example of a useful result from experimental psychology is "Fitt's law," which gives a formula for the time required to move a cursor, depending on the distance D and the width W of the target,
Time = C1 + C2 log2(2D / W)where C1 is a constant for reaction time, C2 normalizes the units used, and log2(2D / W) is called the "index of difficulty." Unfortunately, most of the most important things fall outside its scope, such as the fact that switching between keyboard and mouse can slow users down a great deal, and the fact that being confused can consume a great deal of user "think time."
Most engineers know little about what psychologists really do, so here's a little background to help with understanding this paradigm. Cognitive psychologists make theories of cognition, i.e. thinking, and in general, experimental psychologists are concerned with discovering what kinds of theory work well for describing the results of some class of experiments; thus experimental psychologists devise experiments to test theories. To run a test, you need something very specific, not just general features of a high level theory; therefore it is not possible to actually prove a theory, but only to fail to refute it. Variables that might be measured in psychology experiments include the time to learn how to perform some task, the time taken to complete the task, the error rate in performing it, how well what is learned is retained, and how users report feeling about doing the task. Only after a theory has been validated for a particular class of real phenomena can it be used to make predictions that are relevant to real design issues, and those issues must closely match the experimental setup for the predictions to be valid. As a rule, industrial or applied psychologists are concerned with actual applications, while academic psychologists are more concerned with the theories themselves.
A somewhat popular research theme in HCI with a psychological flavor is cognitive modeling; in my opinion, this area is "reductionist" and "cognitivist," in that it (implicitly) assumes that users have definite "mental models" that can be explicitly described, e.g., by rule-based systems (also called production systems), and also that the semantics of commands and displays can be explicitly described, e.g., by some sort of logical formulae. However much of "what users know" cannot be explicitly represented, because it is implicit or "tacit knowledge," that is embedded in particular contexts of use. For example, there are some phone numbers that I only know by their pattern of button pushes; hence this information can only be available in the physical presence of a standard telephone keypad (or an equivalent). The same holds for knowledge about how to play various sports, which is notoriously difficult to convey in words. Indeed, it holds for almost everything of significant human value, including satisfaction, artistic enjoyment, emotions of all kinds, and even intentions. (The term "tacit knowledge" was introduced by the philosopher Michael Polanyi as part of his criticism of logical positivism, which is the most extreme form of reductionism to have ever achieved much popularity; logical positivism is also a basis for much of modern cognitivism, especially in the extreme form embraced by early AI research, which I have dubbed "logical cognitivism.")
In my opinion, to assume that users can be described in the same style as machines is a strong form of reductionism that is demeaning to humans; moreover (and perhaps more convincingly), it does not work very well in practice, except sometimes at the lowest level of mechanical operations, such as predicting how long it will take to type in material already written on paper; but models at such low "keystroke" levels have relatively little practical value.
Another modelling approach is to give a grammatical description of the possible interactions in an interface; this can be useful for exposing certain kinds inconsistency, but it is more applicable to command line interfaces than to GUIs, since they cannot capture the graphical metaphors that make such interfaces appealing (e.g., the ever popular "desktop" metaphor), nor in fact can they capture any significant kind of context dependency. Similar objections apply (though with less force) to Shneiderman's Object-Action Interface (OAI) model, which assumes that interfaces can be expressed as simple trees. It seems to me that, on the contrary, organizing metaphors (like the desktop) cut across such hierarchies, linking objects at quite different levels in complex ways with concepts that are not explicitly represented at all (such as "opening" something). Moreover, "hyperlinks" as in HTML allow arbitrary cyclic graph structures.
Representing user intentions as simple trees seems even more hopeless, and the idea of representing plans by trees (or by even more complex formal structures) has been demolished by Lucy Suchman in her book Plans and Situated Actions, as well as by many other authors in many other places; even for the relatively clear situation of plans for computation, which we normally call "programs," no one today would want to try to get by with a pure branching structure (without loops). However, even an incomplete model can be better than no model at all, because it does highlight a number of issues that must be addressed, and it also provides some support for solving some of them.
The list of advantages and disadvantages of styles of interaction (on pages 71-74 of Shneiderman's third edition) is really useful, especially the summary on page 72. However, I would add to the disadvantages of direct manipulation that it is hard to use for navigating and manipulating large homogeneous structures, such as proof trees. It is also worth remarking that many designers would strongly disagree with the view that "the set of tasks must be determined before design can begin" - the modern alternative called iterative design involves (among other things) building a series of prototypes, of which the early ones can be very rough, and then successively refining them into a releasable system, by constantly taking account of user feedback. In my experience, attempting an exhaustive task analysis before beginning design is a very bad idea; it will consume a great deal of time, the result is likely to be very inaccurate, and quite possibly will be out of date before it can be deployed.
Direct manipulation occurs when a visual (or other) representation can be manipulated (e.g., by drag-and-drop, or clicking on parts) in a way that corresponds to what is represented. This can be very intuitive and convenient when it works, though there are some interesting cases where it does not work well. Shneiderman has long been a strong advocate of direct manipulation, and has suggested that it is a viable alternative to agents. Shneiderman's text includes a lengthy flame about agents (pages 83-89 of the third edition) emphasizing the problem of responsibility; his position is argued much more effectively here than in the Scientific American interview; see also Agents of Alienation by Jaron Lanier on this same topic, which has given rise to ongoing debates in the HCI community. Box 2.2 on page 84, comparing humans and machines, is fun. I would like to emphasize the following sentence from page 85, which occurs in the context of air traffic control, but which applies equally well to many other domains, such as nuclear reactor control, and the so called Star Wars missile defense system:
In short, real-world situations are so complex that it is impossible to anticipate and program for every contingency; human judgement and values are necessary in the decision-making process.Much more along these lines can be found in the important book Normal Accidents, by Charles Perrow; he shows that many catastrophic accidents, such as the famous Three Mile Island nuclear incident, arise from the simultaneous occurrence of several unusual situations.
Lanier's Agents of Alienation is an unusual piece. In my opinion, Lanier's rhetoric is excessive (though perhaps a refreshing contrast to Shneiderman's rather flat academic style), and he glosses over some important points; but let's seek out what is interesting in it. For now, I would highlight two main points: (1) Lanier is against agents, and does not accept that they "are inevitable" (a position he attributes to Negroponte); (2) Lanier goes beyond purely technical issues, and raises basic ethical issues - in this respect, his argument against agents is quite different from Shneiderman's. However, he does connect with issues in user interface design, and it is very interesting to notice that his list of agents being promoted in 1995 is now a list of notorious failures. See also Direct Manipulation vs. Interface Agents, by Ben Shneiderman and Pattie Maes, Interactions, 4, no. 6, pp 42-61, 1997, a digest of a debate held at CHI 97, which attracted a lot of attention in the HCI community. Some further comments and insights on this debate are given in my paper Are Agents an Answer or a Question?
On page 85 Shneiderman says: "The entire system must be designed and tested, not only for normal situations, but also for as wide a range of anomalous situations as can be anticipated." A related sentence from page 89 says: "Extensive testing and iterative refinement are necessary parts of every development project". It's amazing how often engineers think they can get things right the first time - and then end up wasting a lot of time as a result. I went so far as to have a banner printed to hang in my own lab, saying "Iterative Development!" during the programming of our BOBJ and Kumo systems. Shneiderman's summary for practitioners (section 2.10, page 89) is very good, but his agenda for researchers (section 2.11, page 90) calls for projects that seem to have an overly reductionist flavor; moreover, his phrasing hints that he somewhat doubts the value of cognitivist oriented research. (A strong point of Shneiderman's book is its human, humane, and humanistic tone, of which an emphasis on diversity is but one example. Shneiderman really believes that good user interface design can and should help improve the lot of humanity, e.g., by providing users with various disabilities better access to a wide variety of resources.)
User participation can be difficult, frustrating and even misleading, but it is usually essential. In particular, various prototyping techniques can be used to get user feedback very early in the design or even requirements phase of a project. This is one place where social issues can enter in an obvious and significant way. An important point is that asking users why they do something often does not work: much of user knowledge is tacit rather than explicit; moreover, users often have not previously thought about issues that designers want to raise, and even if they have, they may have a biased, incomplete, or even mistaken point of view.
The following are three important general lessons from our discussion of methodology: