Since this class is not using Shneiderman as a text, you can just skim sections 4.2 and 4.3 below, extracting the content that is relevant to this year. Notes on other readings will eventually be added.

This paper was written for a conference in the area of software engineering that is called requirements engineering, which is concerned with determining the properties that a system should have in order to succeed when it is built. Obviously, the user interface is one area of concern for requirements, and in fact, everything in this paper applies to user interface design as a special case; in general, you can just substitute the phrase "user interface design" for "requirements engineering" to get a paper that reads as if it were written for this course. The paper goes much deeper into the theories behind interviews, questionaires, focus groups, etc. than our text does, and in particular, it provides theoretical tools for understanding why various techniques are unlikely to work in certain situations.

One concept of particular relevance to user interface design is that of adjacency pair, because it can be used in exploring the naturalness of many dialogue-like featues of interactive systems, such as login and logout procedures for operating systems. Moreover, the related notion of noticeable absence can be used to explain why an interaction with a computer system seems unnatural when the systems fails to produce second part of an adjacency pair after the user has produced the first part. Note that login-logout is not an adjacency pair, because a logout is not necessarily expected after a login, and even if it does occur, may be separated by a very large time lapse; in particular, the lack of a logout is not a noticeable absence, thus confirming its non-participation in an adjacency pair.

I would like to emphasize that there are many interesting instances of adjacency pairs in computer interfaces. For example, in UNIX when we say we want to erase a file, if we have properly redefined the "rm" command, then we will be asked if we really want to remove each file, and we must respond in order for the action to be accomplished. Logging in and logging out each provide an example of an adjacency pair. However, there are many examples of such pairs in graphical systems where the design is poor; for example, the user may be forced to acknowledge the occurrence of events that are basically trivial by clicking on "OK," or may be annoyed by popup windows that contain distracting or even irrelevant information. The careful empirical studies in ethnomethodological conversation analysis of how adjacency pairs are organized in ordinary conversation can provide designers with many useful examples, and with hints on how to do better.

This chapter doesn't give enough detail on various methods to make it clear what you really have to do, but it does give a good overview, showing why such methods are needed, what they do, and (to an extent) how to choose among them. A more detailed overview and comparison of certain methods is given in Techniques for Requirements Elicitation.

For expert reviews, it is important to clarify what kind of expert you have: domain experts will be useful in a completely different way than interface design experts. For example, expert cognitive walkthroughs may have little value in evaluating how new users will respond, but they can still be useful, e.g. for catching inconsistencies.

Usability labs are not available for most projects. The thinking aloud method has serious problems with tacit knowledge, and also produces unnatural discourse that can be hard to analyze (for details see Techniques for Requirements Elicitation, where this method is called "protocol analysis"). Analyzing videotapes is hard work but can be worth it for some problems if done well. Paper mockups are often a good idea, because they are so easy; PowerPoint (or other) slides are one step up from that. The limitations of usability testing mentioned on p.132 are important.

The advice on surveys is good but too vague, and the suggested questions on p.133 are also too vague; Table 4.1 is better. As Shneiderman says, "If precise - as opposed to general - questions are used in surveys, then there is a greater chance that the results will provide useful guidance for taking action" (p.134). Statistics really should be used to analyze the results of surveys, and this requires rather sophisticated knowledge. Acceptance tests are actually part of the contracting process, but user interface professionals may become involved in their design.

Interviews and focus groups are highly recommended (at least by me). Continuous data logging raises serious privacy issues; moreover, the social information that might have the most impact is probably not available from such a source. Bulletin boards, newsletters, online help, etc. can be important. As Shneiderman says, "Every technical system is also a social system that needs to be encouraged and nurtured" (p.149). Controlled psychological experiments are a lot of trouble and unlikely to be very useful in most cases. Maybe the sentence on p.150, "If you are not measuring, you are not doing human factors!" is left over from a previous edition, as there is little in this book that argues for the value of human factors in this traditional sense, which is at most a small part of what user interface designers do today.

To read this material, you should be familiar with BNF and with Unix command specification notation; it will also help if you familiar with transition diagrams. The purpose of this chapter is to survey tools and techniques for building interfaces. One approach is to have a special language for specifying the interface to be built, from which the software is then generated. This is difficult, in particular because of difficulties in describing interaction.

Statecharts are a significant improvement on standard transition diagrams, but dont really solve, or even address, the hard parts of UID. To me, UAN looks rather ugly. It is not even clear that specifications are necessary for GUIs; for example, JavaScript is nearly terse enough to be considered a specification language, and it is executable. As Shneiderman says, "The World Wide Web is such a powerful force that web-oriented tools are likely to have the brightest future" (p.168); in his inimitable deadpan style, he also says

The rapid pace of change on the Internet is stimulated by the easy sharing of code and the capacity to build quickly on top of the work of other programmers. The frenzy is sometimes alarming, but is usually irresistible.

Interface building tools are extremely important, and again web-based tools seem most important. To me, JavaScript looks superior to the other tools discussed in chapter 5, though Tcl/Tk is also nice.

XML seems a promising solution for many situations, but it is not going to live up to some of the hype I have seen about "putting semantics onto the web" and "making the web one giant universal database." For example, it is good for B2B workflows, and standardized database export formats.

The experiment reported on p.179 is a good lesson in consistency. The "Researcher's Agenda" on p.181 is worth reading over several times; the UCSD Tatami project is trying to deal with several of the issues mentioned here; however, we do not consider metrics a promising research area.

An important conclusion is that there are no good methods for specifying the interactions mediated by GUIs. One reason is that the styles and supporting technologies used are evolving much faster than the theories. Another reason is the inherent creativity and complexity of graphical media. It often happens that the single most useful thing to do is post copies of all the most important screens on a large wall!