I would especially emphasize points (1) and (4); see also Shneiderman's discussion of analogy in the last paragraph of section 6.3.1 (p.205). That point (1) for direct manipulation is an analogy or metaphor is very relevant for us, because it says that direct manipulation involves a semiotic morphism. The physical nature of this metaphor (in (4)) makes it more direct and concrete, and thus easier for users to grasp and to apply. Leibniz, who was no doubt thinking of mathematical notation, makes an important point when he says (quoted on p.185 of Shniederman):

In signs, one sees an advantage for discovery that is greatest when they express the exact nature of a thing briefly and, as it were, picture it; then, indeed, the labor of thought is wonderfully diminished.

It is very interesting to notice that classical semiotics can also give us deeper insight into the nature of direct manipulation, seeing it as an indexicality of motion, often reinforced by a specific kind of iconicity, called diagramatic iconicity by Peirce, where the geometric structure of the sign corresponds to the structure of its object (geographic information systems are particularly clear examples, since the structure of a geographic map corresponds to the structure of some part of the surface of the earth).

In this chapter, Shneiderman often confuses the essentially semiotic nature of direct manipulation with the technologies (or in more semiotic terms, the "media") that are used to implement it. Our semiotic conception of direct manipulation allows us to avoid this error, by clearly distinguishing between what functionality is preserved, and how it is represented. For example, it is perfectly possible to have a virtual reality interface to plain old 1978 DOS, complete with a haptic clicking keyboard and a virtual ancient VT100 screen with bright glowing green characters floating in space before you! Despite the fancy technology, this is still just command line DOS. This confusion is really just one aspect of a larger confusion, between the device that supports an interface, and the design and software that make the device actually function as an interface. Journalists often focus on the physical device (the "box") without giving much thought to the design of the interfaces of the applications that it supports. This is no doubt due in part to their receiving press releases from manufacturers and pressure from the advertising department, but it also reflects a bias in our culture.

Design errors generally show up as violation of the underlying metaphor of a direct manipulation interface, or more generally, for any interface, of its semiotic morphism. One infamous example is the Apple Macintosh use of the trashcan for ejecting a floppy disk; it has confused generations of users, and it violates the trashcan metaphor in that the floppy is not trash. A more complex example discussed in class is the use of lemmas in proofs, which leads to violations of a tree metaphor, but can be patched by using hypertext links (as in Kumo). Another example is the little arrows at the top and bottom (or left and right) of many scrollbars, because the physical motion metaphor does not suggest that these should be "hot." In fact, a scrollbar with this capability is actually a blend of two metaphors, and hence is a bit more difficult to learn; the second metaphor is the same as that for the "up" and "down" buttons on elevators.

The list of problems with direct manipulation on p.204 is very important; please read it at least three times. The "tatami" project in my lab ran into some of these problems with a direct manipulation interface that we built for proofs; it turned out that displaying the proof tree was useless for large proofs, because of the size and homogeneity of the display. Shneiderman's campaign on behalf of direct manipulation has been so successful that today, one is perhaps more likely to see it misapplied than to see it not applied when it should have been.

Piaget's work is in many ways outdated, so the material on page 207 should be taken with some grains of salt. The discussion of WIMP (p.207) is amusing but not very substantive. The guidelines for icon design (pp.208-9) are superficial but suggestive and useful. The remarks about emacs (p.210) are kind, but do not do justice to that amazingly flexible tool.

The last paragraph of section 6.5 (p.213) is interesting in connection with Lanier's piece; please reread Lanier, and see whether you think Shneiderman might not be saying the same thing in his very different way. The remark about notations for representing relations among residents of a home (p.217) seems a bit off the wall, but is interesting to think about from an ethical point of view; consider Bill Gates's high-tech house. The material on virtual environments may sound far out now, but I believe it will become increasingly important (pp.221-228). It is interesting to note that both the term "virtual reality" and the data glove (p.227) were invented by Lanier. The contrast between "immersive" and "looking-at" experience (bottom p.222) is interesting. Augmented reality (p.225) already has important industrial applications (e.g., at Boeing) and no doubt will have more. Situatedly aware shopping carts (p.225) do not appeal to me.

Despite the mathematical character of the formal definitions of sign system and semiotic morphism, these concepts can be used very informally in practice, just as simple arithmetic is used in everyday life. For example, to see if we have enough gas left to drive from San Diego to Los Angeles, we make some assumptions, use some approximations, and only do the divisions and multiplications roughly. It would not make much sense to first work out an exact formula taking account of all contingencies, then do a careful analysis of the likelihoods, and finally calculate the mean and variance of the resulting probability distribution (though this is the sort of thing that NASA does for space shuttle missions). In user interface design, our goal is often just to get a rough understanding of why some design options may be better than others, and for this purpose, assumptions, approximations, and rough calculations are sufficient, especially when there is time pressure.