Chapter 4 of Shneiderman doesn't give enough detail on various methods to make it clear what you really have to do, but it does give a good overview, showing why such methods are needed, what they do, and (to an extent) how to choose among them. A more detailed overview and comparison of certain methods is given in Techniques for Requirements Elicitation.
For expert reviews, it is important to clarify what kind of expert you have: domain experts will be useful in a completely different way than interface design experts. For example, expert cognitive walkthroughs may have little value in evaluating how new users will respond. But this can still be a useful method.
Usability labs are exceptional facilities not available for most projects. The thinking aloud method has serious problems with tacit knowledge, and also produces unnatural discourse that can be hard to analyze (for details see Techniques for Requirements Elicitation, where this method is called "protocol analysis"). Analyzing videotapes is hard work but can be worth it for some problems if done well. Paper mockups are often a good idea, because they are so easy; PowerPoint slides are one step up from that. The limitations of usability testing mentioned on p.132 are important.
The advice on surveys is good but too vague, and the suggested questions on p.133 are also too vague; Table 4.1 is better. As Shneiderman says, "If precise - as opposed to general - questions are used in surveys, then there is a greater chance that the results will provide useful guidance for taking action" (p.134). Statistics should really be used to analyze the results of surveys, and this requires rather sophisticated knowledge. Acceptance tests are actually part of the contracting process, but user interface professionals may become involved in their design.
Interviews and focus groups are highly recommended (at least by me). Continuous data logging raises serious privacy issues; moreover, the social information that might have the most impact is probably not be available from such a source. Bulletin boards, newsletters, online help, etc. can be important. As Shneiderman says, "Every technical system is also a social system that needs to be encouraged and nurtured" (p.149). Controlled psychological experiments are a lot of trouble and unlikely to be very useful in most cases. Maybe the sentence on p.150, "If you are not measuring, you are not doing human factors!" is left over from a previous edition, as there is little in this book that argues for the value of measurement. In any case, human factors in this traditional sense is at most a small part of what user interface designers do today.
One concept of particular relevance to user interface design is that of adjacency pair, because it can be used in exploring the naturalness of many dialogue-like featues of interactive systems, such as login and logout procedures for operating systems. Moreover, the related notion of noticeable absence can be used to explain why an interaction with a computer system seems unnatural when the systems fails to produce second part of an adjacency pair after the user has produced the first part. Note that login-logout is not an adjacency pair, because a logout is not necessarily expected after a login, and even if it does occur, may be separated by a very large time lapse; in particular, the lack of a logout is not a noticeable absence, thus confirming its non-participation in an adjacency pair.
I would like to emphasize that there are many interesting instances of adjacency pairs in computer interfaces. For example, in UNIX when we say we want to erase a file, if we have properly redefined the "rm" command, then we will be asked if we really want to remove each file, and we must respond in order for the action to be accomplished. Logging in and logging out also provide examples of adjacency pairs. However, there are many examples of such pairs in graphical systems where the design is poor; for example, the user may be forced to acknowledge the occurrence of events that are basically trivial by clicking on "OK," or may be annoyed by popup windows that contain distracting or even irrelevant information. The careful empirical studies in ethnomethodological conversation analysis of how adjacency pairs are organized in ordinary conversation could provide designers with much many useful examples and inspiration for how to do better.
The rapid pace of change on the Internet is stimulated by the easy sharing of code and the capacity to build quickly on top of the work of other programmers. The frenzy is sometimes alarming, but is usually irresistible.As if to illustrate this rapid rate of change, this book contains little on internet based technologies, such as HTML, web design, browser design, etc.
The experiment reported on p.179 is a good lesson in consistency. The "Researcher's Agenda" on p.181 is worth reading over several times; the UCSD Tatami project is trying to deal with several of the issues mentioned here; however, we do not consider metrics a promising research area.
One important conclusion is that there are no good methods for specifying the interactions mediated by GUIs. One reason is that the styles and supporting technologies used are evolving much faster than the theories. Another reason is the inherent creativity and complexity of graphical media. It actually often happens that the single most useful thing to do is post copies of all the most important screens on a large wall!