CSE 271 Class Notes 7 CSE 271: User Interface Design: Social and Technical Issues
7. Narrative and Semiotic Morphisms

Section 7.1 assumes that you have read about the basics of Labov's structural theory of narrative, in Notes on Narrative and The Structure of Narrative. Section 7.2 gives some important technical definitions that go beyond those in An Introduction to Algebraic Semiotics, with Applications to User Interface Design

7.1 Applications Involving Narrative

We can understand (some aspects of) narrative structure in terms of semiotic morphisms. For example, books on screen writing by Syd Field prescribe a precise (but naive) dramatic structure for the plots of Hollywood movies: they should have three acts, for setup, conflict, and resolution, with "plot points" that move action from one act to the next. We can describe this "Syd Field structure" using a very simple sign system with a main constructor that builds a thing of sort "plot" from three things of sort "act", and we can then check whether a given film has this structure by seeing whether there is a semiotic morphism from the (structure of the) film to this sign system.

It is clear that as internet bandwidth grows, video will be used more and more; moreover, complex interactive websites are aready common, many with story lines. Techniques for drama apply directly to such cases, but are also much more broadly applicable, in suggesting ways to make displays (i.e., "texts" in the very general sense of signs that are to be interpreted by users) more interesting. My own favorite examples of this come from mathematics. Probably we have all had the experience of trying to read a proof and being frustrated, at least initially, by not being able to see where it was going, or why it was structured as it was. This is often because the proof author did not describe the difficulties that arose in constructing the proof, but instead just described the machinery that was constructed to overcome the difficulties. For example, one often comes upon an assertion in the middle of a proof whose relation to the main result to be proved is far from clear. It will help if the proof author has broken this assertion out of the main text and labelled it a "lemma," but even then, it is often far from clear why it will be needed later on. This occurs because the values of professional mathematicians are quite different from those of students, and of most users of computer-based theorem provers: professional mathematicians like to demonstrate their technical dexterity with proofs that surprise readers (see Reality and Human Values in Mathematics).

The "Syd Field" structure might suggest first showing how a proof fails without a lemma (in dramatic terms, this gives rise to a conflict), and then showing how it goes through with the lemma - the "first act" will of course set up the proof, including what is to be proved and what is assumed. We used this structure in proving a simple property of flags on our Tatami project website (you may ignore the proof details, and look at the explanation for why the first attempt at the main result fails, motivating the second attempt, which succeeds with a lemma).

It is also interesting to consider the problem of translating from one language to another. This has been much studied in computer science, and with today's increased processing and memory power, is finally bearing some fruit (though sometimes that fruit may seem a bit raw - e.g. the translation option on websearch engines often gives results that are amusing and/or depressing). Semiotic morphisms illuminate some of the difficulties. It will help bring out the issues if we focus on the particularly difficult case of poetry, noting that less severe versions of the same difficulties can arise in any form of language. First, notice that poems may have, or fail to have, many different kinds of structure; for example, they may or may not be divided into stanzas, have a fixed meter, or rhyme; they are usually divided into lines, but even this does not always hold. Moreover, some poems have a geometrical form that is of interest (e.g., poems by e e cummings), and many different conventions are used for punctuation. In general, poems simultaneously exhibit more than one structure. All of this is easily expressed using sign systems, with various kinds of constructors and relations. For example, iambic pentameter is a particular sign system for syllabification. Sonata form plays a similar role in music from the classical period.

Secondly, notice that structures are often more natural for one language than for another. For example, it is much easier to rhyme in Romance languages like Spanish and Latin than in English. From this, it follows that it may be desirable to fail to preserve certain features, such as rhyme, when translating across languages; and it may even be desirable to add some feature to the translation that was not present in the original, e.g., adding rhyme when translating from Chinese to Spanish. Of course, every feature may be preserved to a certain extent, rather than being either fully preserved or not preserved.

Many people agree that mathematical proofs are an area where better design could have a big impact, e.g., in K12 education. The above ideas provide a way to explore how computer technology might help with this. Of course, there are many other design areas where making the content more dramatic could help, such as textbooks, courses, and manuals; private sector media already make extensive use of drama in their work, e.g., look at TV newscasts, or "real crime" documentaries, or other "reality" programming. But it should not be thought that drama is the only issue in narrative; the extensive literatures on creative writing and the novel raise many other issues. Also, there is the structure of oral narratives of personal experience that we studied in The Structure of Narrative, due to the linguist William Labov and others, and there is a famous discussion of the semiotic structure of Russian fairy tales due to Vladimir Propp. The main point of the present discussion is to claim that any instance of such a structure can be seen as arising through a semiotic morphism to a sign system that embodies the given structure. A text can then be seen as a blend of all the structures that are involved. I also claim that being aware of this viewpoint can help designers in their practical work, if they are also aware of the variety of narrative structures that are potentially applicable.

It is interesting to see how our theory of semiotic morphisms solves a problem that has been noticed by many theorists of narrative structure, which is that texts often fail to include some of the features that are supposed to be part of the generic structure; even important features are sometimes left out. This corresponds to the fact, with which we are already familiar, that semiotic morphisms can be partial, i.e., they can fail to preserve some aspects of source signs. Moreover, the fact the target is a sign system, not just some fixed given structure, means that many different structures, perhaps with a variety of substructures, etc., can be possible, not just some single fixed structure. This allows a great deal of flexibility. For example, the Labov structure permits arbitrary sequences of narrative clauses, and allows many different kinds of evaluative material. Moreover, its opening and closing sections are optional. Artists often play with structure to create interesting effects; for example, false endings in classical symphonies.

The Labov structure can be applied in many different ways. For example, user manuals for computer systems often describe sequences of steps, and these may be less tedious to read if they are given a more narrative-like structure, although this can of course be overdone, with unpleasant results, as we previously saw with humor and cuteness. The approach here is to consider semiotic morphisms from the Labov narrative structure sign system.

7.2 Notes on Section 3 of An Introduction to Algebraic Semiotics, with Applications to User Interface Design

First, recall that the composition (f;g) of semiotic morphisms f: T -> T' and g: T' ->T'' is defined, for x an element of T, by (f;g)(x) = g(f(x)). This means first apply f to x, and then apply g to the result; the semicolon notation is borrowed from programming languages, where it again indicates first do one statement, then the next.

Definition: A binary relation > on a set P is a partial ordering if it is transitive (i.e., a > b and b > c imply a > c, for all a,b,c in P), and is anti-reflextive (i.e., a > a does not hold for any element a in P). A partial ordering > is a total ordering if for all a,b in P, either a = b or a > b or b > a.

Notice that the so-called "unordered list" of HTML actually produces graphic elements that display a total order in a natural way (since for each pair of distinct list elements, one is necessarily above the other); HTML "unordered lists" differ from "ordered lists" in being unenumerated, not in being unordered.

Definition: Given two partially ordered sets, P with > and P' with >', their lexicographic product consists of the set of pairs (a,a') with a in P and a' in P', ordered by (a,a') > (b,b') if a > a' or (a=a' and b > b').

Theorem: If P and P' are both totally ordered, then so is their lexicographic product.

The reason that the TOD ("time of day") semiotic space has some odd looking representations that appear to be good mathematically is that this particular theory of time is very basic, and does not include certain social conventions which we expect to see preserved in our representations of time. The most important of these is that the 1440 minutes of a day are enumerated using two counters, one that goes up to 24 and the other up to 60; these are combined by the constructor (_,_), to create pairs of counters. Here are the axioms for this more detailed source space, where h, m are variables for the hour and minute counters, respectively, and s denotes the unary "next" (or "tick") function on time (i.e., on the pairs of counter values), and also denotes the successor function on integers:

s(h, m) = (h, s(m)) if s(m) < 60 . s(h, m) = (s(h), m) if s(m) = 60 and s(h) < 24 . s(23, 59) = (0, 0) . It is interesting to notice that the usual ordering on time is exactly the lexicographic product of the two counters, that is, (h, m) > (h', m') if h > h' or (h=h' and m> m'). With this additional structure on the source space, the allowable semiotic morphisms are what we would expect, and in particular, both the strange unary representation, and the decimal number of elapsed minutes, fail to preserve the structure created by the constructor (_,_).

Definition: A projection M on a semiotic space S is an semiotic morphism with source and target S that is idempotent , i.e., that satisfies the equation

M ; M = M . A simple example maps a list of numbers to their sum, given as a list. For example, this morphism maps (1,2,3) to (6), and (4,5,6) to (15), and (7,8,9) to (24), and then maps each of them to itself. A similar example computes the average of a list of numbers. A bit more complex example is a mapping of lists of lists to numbers lists of sums of the component lists.

In general, a projection can be undefined on many elements of its semiotic space. A simple example is mapping numbers to their remainder modulo (say) 60; it is defined on all numbers, but not on the non-numerical character strings in W. A more complex example is the morphism on W to itself that takes total elapsed minutes to military time; more precisely, if N is a string of decimal digits, then

M(N) = Q : R M(Q : R) = Q : R where "Q : R" is the quotient Q of N by 60, as a string of digits, followed by the colon, followed by the remainder R of N by 60, again as a string of digits.

Definition: A semiotic theory T' is a refinement of a semiotic theory T if there is a semiotic morphism f: T -> T' which preserves all relevant properties of T and which induces an isomorphism of the algebras of terms of T and T'. [[More technically, if G, G' are the signatures of T, T', and if T(G) denotes the algebra of G-terms, then there must be a view f: G -> Der(G') from T to T' (where Der(G') is the derived term signature of G') that induces a G-isomorphism T(G) -> T(G')|G, the reduction of T(G') to a G-algebra via f.]]

For example, the two counter theory for time in minutes is a refinement of the one counter (with cycle 1440) theory. Similarly, the three counter theory of time in seconds is a refinement of the one counter theory with cycle 86,400. In these two examples, the simple theory is refined by encoding some additional social conventions as constructors and axioms, in a way that is consistent with the original theory.

Exercise: Consider the same points that are discussed above for time of day in minutes, but now for time of day measured in seconds, including the three corresponding clocks. Hint: The more refined version of the theory should have three counters instead of just two.

To CSE 271 homepage
To the next section of the class notes
To the previous section of the class notes
Maintained by Joseph Goguen