Algebraic Semiotics
Contents
Short Overview

Algebraic semiotics is a new approach to meaning and representation, and in particular to user interface design, that builds on five important insights from the last hundred years:

Semiotics is the study of signs. Our research attempts to make this area more systematic, rigorous, and applicable, as well as to do justice to its social and cognitive foundations. Algebraic semiotics combines aspects of algebraic specification and social semiotics. It has been applied to information visualization, user interface design, the representation of mathematical proofs, multimedia narrative, virtual worlds, and metaphor generation, among other things. The webnote Semiotic Morphisms is a mostly non-technical exposition of some basics of algebraic semiotics, and the course CSE 271 includes more of the same plus many applications. See also the User Interface Design homepage. More detailed technical information may be found in Semiotic Morphisms, Representations, and Blending for User Interface Design, Foundations for Active Multimedia Narrative: Semiotic spaces and structural blending, and An Introduction to Algebraic Semiotics, with Applications to User Interface Design.

A basic concept is that of a semiotic morphism, which provides representations in one sign system (the target) for signs from another (the source). Semiotic morphisms can be partial, i.e., they do not necessarily have to preserve all of the signs or all the structure of the source system. The degree to which semiotic morphisms preserve various features provides a basis for comparing the quality of representations, and leads to an interesting study of trade-offs.

To illustrate these ideas, consider proofs (in some fixed logical system) as forming a sign system; an extension of this system includes additional information to help with understanding proofs, such as motivation, background tutorials, and examples. Another sign system is given by website technology (HTML, JavaScript, XML, etc.). Then representations of proofs as websites are morphisms from the first system (or its extension) to the second, and the orderings on semiotic morphisms compare aspects of the quality of such representations. Some website design principles, called the Tatami conventions, were extracted from our study and embodied in our Kumo tool, which combines proof assistant and website generation capabilities; it generates so-called "proofweb" data structures that use HTML, JavaScript, etc., which can then be viewed with any browser. For more details, see Web-based Support for Cooperative Software Engineering. Some other applications are discussed in Information Visualization and Semiotic Morphisms and in Steps towards a Design Theory for Virtual Worlds.

The "world famous" UC San Diego Semiotic Zoo contains a collection of semiotic morphisms, each an example of bad design arising through failure to preserve some relevant structure. (Notes: (1) The zoo still has one wing under construction; and (2) it won a "Creativity Award" from Art & Technology.)

Mathematical foundations can be provided by the rather recent and very abstract field called "category theory" (it is not related to the area of psychology of the same name), by noting that sign systems together with semiotic morphisms form a category. Some modest additional axioms are satisfied, which leads to the notion of a 3/2-category. An appropriate notion of colimit for such categories has properties that make it suitable for studying the blending of sign systems, as explained in An Introduction to Algebraic Semiotics, with Applications to User Interface Design. See also Semiotic Morphisms, Representations, and Blending for User Interface Design, to see how hidden algebra extends algebraic semiotics to handle interaction. The more recent papers Foundations for Active Multimedia Narrative: Semiotic spaces and structural blending and Information Visualization and Semiotic Morphisms include more intuitive introductions to many issues, and the webnote Semiotic Morphisms may be a convenient place for many readers to start.


Some Concrete Applications

The following are some projects that used algebraic semiotics in various ways:

  1. Henry Mitchell and Breese Stevens built a website for San Diego Jazz Party, a non-profit organization that organizes a once a year weekend jazz festival, with profits supporting music in the San Diego school system.
  2. Abigail Gray built a website for a network of San Diego animal shelters; her MS thesis documents the use of algebraic semiotics for many design decisions.
  3. Cynthia Bailey Lee wrote a paper on political cartoons for CSE 271 in 2003.
  4. Dana Dahlstrom and Vinu Somayaji wrote the Tutorial on Semiotics for CSE 271 in 2004.
  5. The "world famous" UC San Diego Semiotic Zoo contains a collection of semiotic morphisms, each an example of bad design arising through failure to preserve some relevant structure.
  6. A collection of proof displays generated by version 4 of the Kumo proof assistant and website generator, as part of the Tatami project, a goal of which is to make machine proofs much more readable than is usual.

Social Foundations

Towards a Social, Ethical Theory of Information describes a theory of information based on social interaction. This theory provides a social foundation for algebraic semiotics, in which some ideas from ethnomethodology play a key role. Ethnomethodology is a branch of sociology that studies ordinary natural social interaction; it has especially considered conversation. In this framework, we may define a sign system to be a system of distinctions, grounded in the ordinary practices of some social group, and used for communication within that group. Signs are the "items" that are so distinguished.

Signs and sign systems are grounded in the actual practices of particular groups. Some sign systems are used by analysts of a group, rather than by members of that group. For example, formal grammars are used by linguists to study a language; they are not ordinarily used by the speakers of that language. Of course, analysts also form their own social groups.

Values are an inherent part of any social group, and the concepts (including distinctions) and methods that a group uses naturally reflect those values. Therefore analysts can hope to derive values by observing members' concepts and methods. One might go so far as to say that social groups, values, and communication are coemergent, in the sense that each produces and sustains the others.

Requirements Engineering as the Reconciliation of Technical and Social Issues discusses situated abstract data types, which are a precursor of our current algebraic notion of sign system, with a greater emphasis on social context, and with some examples showing how representation interacts with social context. See also Reality and Human Values in Mathematics, which applies discourse analysis (in the sense of sociolinguistics), cognitive linguistics and ethnomethodology to mathematical discourse, showing how the reality of mathematical objects is achieved, and the role of values in this process; a pdf version is also available.


Brief Annotated Bibliography Background information on algebraic specification and category theory can be found in the following:
Other Links
Maintained by Joseph Goguen
To the research projects index page
Last modified: Wed Mar 30 08:12:59 PST 2005