Communication is always mediated by signs, which always occur in systems of related signs. Often signs can be seen as representations of other signs, for example, in user interface design. This note sketches a framework for studying representation, and in particular, for considering what makes some representations better than others. It should be considered an introductory exploration of a new path that may lead to an improved theory for areas such as user interface design.

The approach is based on precise notions of *sign system* and of
*semiotic morphism*, the latter being a systematic translation between
sign systems. The theory is intended to apply to aspects of communication,
such as generating explanations, coordinating material in multiple media, and
comparing the effectiveness of representations, including metaphors and user
interface designs. Though transformations are fundamental in many areas of
mathematics and its applications (e.g., linear transformations, i.e.,
matrices), they seem not to have been previously studied in semiotics, where
they can be seen as conveying "meaning," or more precisely, as translating
between levels of representation. Semiotic morphisms are proposed as a new
fundamental concept for both semiotics and for user interface design.
Philosophically, we have been concerned to avoid the usual Platonistic
approach to signs.

This on-line document started life as an experiment to convert a LaTeX paper to HTML and edit it for use my class on user interface design; the paper from which it started, "Extended Abstract on Semiotic Morphisms," was written in 1995, and later evolved into [B]. In the editing process, I attempted to preserve the brief and very informal style of the original, and to convey the motivation for the formal approach that is expounded elsewhere, with some emphasis on applications to user interface design.

An adequate survey of semiotics would take several volumes, and an adequate survey of multimedia, instructional aids, graphics, metaphor theory, and user interface design would surely take far more. Consequently, only a few items that especially influenced our thinking are mentioned here. First, this research arose from joint work with Charlotte Linde done 20 years ago [9]. The definition of sign system given here builds upon the classical work by the American logician Charles Sanders Peirce [13] and the Swiss linguist Ferdinand de Saussure [3]; the latter saw the importance of viewing signs as participating in systems defined by differences, while the former sought ways to classify the immediacy with which signs convey their meanings.

More recently, analogies and file names were studied by Gentner [5] and Carroll [2] respectively; their formalisms are similar to
ours in emphasizing structure over content, but lack important features like
constructors, levels and axioms; they are based on models rather than
theories. Lakoff, Johnson and others [12] have studied the structure of metaphors
in detail, while Fauconnier, Turner and others have studied the important
concept of "blending" [4].
Sacks' notion of "category system" [16]
from ethnomethodology [17] is also related,
though it is informal and lacks detail. Recent work of the author on the
nature of information [8] also uses ideas from
ethnomethodology, and can be seen as providing a philosophical and
methodological foundation for the present work. A much more complete
exposition of the technical material of this paper may be found in [B], and the most recent information can be found
on the Algebraic Semiotics Homepage and
the website of the course **CSE 271**.
The insight that studying *errors*, i.e., *badly designed* sign
systems, can help to better understand what it means for a sign system to be
well designed is pursued in the exhibits of the **UC
San Diego Semiotic Zoo**. Both the foundations and the applications of
this subject are growing very rapidly; the What's New page of the author's website is a good place to look for the latest
results.

As the linguist Edwin Sapir liked to say, "all systems leak," i.e., every
theory has some gaps, some phenomena that it does not adequately cover. But
surely it is better to have a precise description that is somewhat wrong, than
to have a description that is so vague that no one can tell if it is wrong.
This paper makes no attempt to formalize actual living meanings; rather, we
wish to provide better means for expressing partial understandings more
exactly. We focus on the *structure* of systems of signs, which is
more amenable to formalization than are basic signs or the uses of signs.
Precision is also needed as a basis for computer programs that apply the
theory. It should not be thought that we believe there are actual existing
ideal Platonic mathematical entities that correspond to signs, nor that we
believe signs are self-existing real entities in the world, as seems to be
presupposed by most of the literature on semiotics.

Despite the formal mathematical character of the definitions of sign system and semiotic morphism (these are actually given in reference [B], and are only discussed informally in this note), these concepts can be used very informally in practice, just as simple arithmetic is used informally in everyday life. For example, to see if we have enough gas left to drive from San Diego to Los Angeles, we make some assumptions, use some approximations, and only do the divisions and multiplications roughly. It would not make much sense to first work out an exact formula taking account of all contingencies, then do a careful analysis of their likelihoods, and finally calculate the mean and variance of the resulting probability distribution (though this is the sort of thing that NASA does for space shuttle missions). In user interface design, the goal is often just to get a rough understanding of why some design options may be better than others, and for this purpose, assumptions, approximations, and rough calculations are sufficient, and are especially suitable when there is time pressure.

19 October 1996

Revised 5 February 2000, and further edited in May 2000 and May 2001. Additional minor edits in April 2004.