User Interface Design
Contents
Short Introduction
New technologies provide the means to build superb new systems, as well
as phenomenally ugly and awkward systems that still fully meet performance
and functional requirements. There is a notable lack of adequate
theoretical foundations in the area of interface design. Our research is
aimed at closing this gap, and is exploring new approaches to a scientific
understanding of basic issues of usability, representation, coordination,
and value that arise in interface design and related areas, such as how to
best display complex scientific information in multimedia; there is also
some focus on distributed cooperative work and on formal representation.
Techniques include ideas from algebraic semantics, cognitive science and
semiotics, for structure, interaction, metaphor, and blending. For more
detail, see the Short Summary of Algebraic Semiotics
below, the Algebraic Semiotics
homepage, publications in the Brief Annotated
Bibliography below, and also the UCSD courses CSE 271 (graduate level) and CSE 171 (undergraduate level). Our most recent work
places more emphasis on the direct integration of values with the entire
design process; see the draft introduction to the book in progress Value-Driven Design, with Algebraic
Semiotics, by Joseph Goguen and Fox Harrel; a draft outline of the whole book
is also available.
Brief Annotated
bibliography
- Semiotics, Compassion
and Value-Driven Design, by Joseph Goguen. Slides for keynote
address at conference on Sociology of Informatics, Kyoto University, 13-14
December 2005; discusses semi-formal application of algebraic semiotics to
software design, emphasizing the roles of values and compassion. For more
information, see also the older but related papers: Semiotics, Compassion and Value-Centered
Design, in Proceedings, Organizational Semiotics
Workshop, University of Reading, UK, 11 - 12 July 2003; and Groundlessness, Compassion and
Ethics in Management and Design, in Managing as Designing,
edited by Richard Boland and Fred Callopy, Stanford University Press, 2004,
pages 129-136.
- Griot System Homepage describes the
multimedia display/performane engine we are developing, to integrate
generative graphics, structured generative text (narrative, poetry, etc.),
music, and so on. The page describes the system architecture, lists
performances using the system, and gives links to references providing more
details.
- Information about our performance, The
Griot Sings Haibun on 28 October, celebrating the opening of the
California Institute for
Tececommunications and Information Technology building, in cooperation
with the UCSD Center for Research in
Computing and the Arts. The performers were: Bertram Turetzky,
contrabass; David Borgo, saxophones & flutes; Ryoko Amadee Goguen, piano;
Joseph Goguen, direction, poetry performance, and polypoem source; D. Fox
Harrell, Griot implementation and polypoem execution. Links to Program for the whole concert and Photo of participants in concert (plus
some family).
- Formalization and Implementation for
Cognitive Semantics, slides for keynote address at Workshop on the Potential
of Cognitive Semantics for Ontologies, Torino Italy, 3 November
January 2004. Sorry, it is in MS Powerpoint.
- Homepage for Computational Narratology, with
links for work with Fox Harrell in this exciting new area, which is
concerned with developing theory and technology to produce interesting new
multimedia narratives on the fly for computer games, active poems, and
other new media objects.
- Style as Choice of Blending
Principles, by Joseph Goguen and Fox Harrell. In Style and Meaning in
Language, Art, Music and Design, ed. by Shlomo Argamon, Shlomo
Dubnov and Julie Jupp; Proceedings of a Symposium at 2004 AAAI Fall
Symposium Series, Technical Report FS-04-07, AAAI Press, 2004, pages 49
to 56 (Washington DC, October 21-24). There are also a postscript version, and powerpoint slides for the lecture. This paper
proposes a new approach to style based on the principles for blending that
works employ; it also includes an implementation approach to syntax based
on structural blending and cognitive grammar, and proposes a
reconsideration and generalization of optimality principles for blending.
A poetry generation system based on this ideas is also explained, and some
output is included.
- Foundations for Active Multimedia
Narrative: Semiotic spaces and structural blending, by Joseph
Goguen, and Fox Harrell. To appear in Interaction Studies: Social
Behaviour and Communication in Biological and Artificial Systems.
Algorithms and models for new genres of gaming, entertainment, and art,
based on theories of blending, narrative, algebraic semantics and
semiotics; also reports experiments with active poetry. There is also a postscript version.
- Steps towards a Design Theory for
Virtual Worlds, by Joseph Goguen. Chapter in Developing Future
Interactive Systems, edited by Maribel Sanchez-Segura, published by
Idea Group, pages 116-152, 2005; an html
version is also available. This paper is an expanded and revised
version of a keynote lecture given at the Virtual Worlds and Simulation
Conference, Phoenix AZ, 10 January 2001, and published in
Proceedings, Conference on Virtual Worlds and Simulation, edited by
Christopher Landauer and Kirstie Bellman, Society for Modelling and
Simulation, 2001, pages 298-303. It sketches algebraic semiotics and its
applications, especially to user interface design and scientific
visualization; it is an update of a "distinguished lecture" Algebraic
Semiotics and User Interface Design, given at the Institute for
Software Research, University of California at Irvine, 20 October 2000.
- Semiotic Morphisms, Representations, and
Blending for User Interface Design, by Joseph Goguen. Keynote
lecture, in
Proceedings, AMAST Workshop on Algebraic Methods in Language
Processing, AMAST Press, 2003, pages 1-15 (held Verona, Italy, 25 -
27 August 2003). This paper extends algebraic semiotics to handle
interaction by using hidden algebra; some examples are given in detail. A
post script version of the paper, and the slides for the talk are also available
(you may have to change the orientation to swap landscape).
- Information Visualization and
Semiotic Morphisms, by Joseph Goguen and D. Fox Harrell. An informal
introduction to the notion of semiotic morphism from the field of algebraic semiotics, showing how information visualization,
in both analysis and design, can benefit from a viewpoint based on
structure-preserving morphisms. To appear in Multidisciplinary Approaches
to Visual Representations and Interpretations, ed. Grant Malcolm (Elsevier
2004), pages 93-106. This book consists of revised versions of papers from
the Second International Conference
on Visual Representations and Interpretations, Liverpool UK, 9-12
September 2002. An older html version
of the paper is also available. Further background for this paper can be
found in the webnote Semiotic
Morphisms and the paper On
Notation.
- Web-based Support for Cooperative
Software Engineering, by Joseph Goguen and Kai Lin. In Annals of Software Engineering, volume
12, No. 1, pages 167-191, 2001; a special issue on multimedia software
engineering, edited by Jeffrey Tsai. This is an overview of the Tatami
project, featuring version 4 of the Kumo proof assistant and website
generator, and focusing on its design decisions, its use of multimedia web
capabilities, and its integration of formal and informal methods for software
development in a distributed cooperative environment. The paper is a revised
and expanded version of a paper by the same authors which appeared in the
Proceedings, International Symposium on Multimedia Software
Engineering, edited by Jeffrey Tsai and Po-Jen Chuang, IEEE Press, pages
25-32; the meeting was held in Taipai, Taiwan, December 2000. A gzipped postscript version is also available.
- Are Agents an Answer or a
Question? by Joseph Goguen. An essay on agent technology, analyzing
the historical, social and ethical context of agent research, with a critique
of agent technology based on the social behavior of real (human) agents. A version with more digressions and more complete
references appeared in Proceedings, JSAI-Synsophy International Workshop on
Social Intelligence Design, held 21-22 May 2001, Matsue, Japan; also
available in a pdf version.
- An Introduction to Algebraic Semiotics, with
Applications to User Interface Design, by Joseph Goguen, in
Computation for Metaphor, Analogy and Agents, edited by Chrystopher
Nehaniv, Springer Lecture Notes in Artificial Intelligence, volume 1562,
1999, pages 242-291. This is the original paper on the mathematical
foundations of algebraic semiotics, with 3/2-categories and 3/2-colimits,
which provide a foundation for blending with any optimality principle given
as a partial ordering on morphisms; the paper also has many examples,
especially from user interface design. A preliminary version appeared in
Proceedings, Conf. on Computation for Metaphor, Analogy and Agents
(Aizu-Wakamatsu, Japan, 6-10 April 1998) pages 54-79, an earlier version of
which is Semiotic Morphisms, Technical Report CS97-553, August 1997.
A now obsolete extended abstract appeared in Proceedings, Conference
on Intelligent Systems: A Semiotic Perspective, Volume II (National Institute
of Standards and Technology, Gaithersberg MD, 20-23 Oct 1996) pages 26-31.
- Visit the "world famous" UC San Diego Semiotic Zoo for an astonishing collection of
exotic semiotic morphisms, each an example of bad design arising through
failure to preserve some relevant structure. (Notes: (1) The zoo is
still under construction, and one wing is currently closed for rennovation;
(2) the zoo has won a "Creativity Award" from Art & Technology.)
- Notes on Narrative, by
Joseph Goguen. Brief overview of some techniques for the analysis of stories,
including summaries of the structural theory of narrative, and techniques for
the extraction of value systems from stories. Somewhat edited in May 2001.
- Towards a Social, Ethical Theory of
Information, in Social Science Research, Technical Systems and
Cooperative Work, edited by Geoffrey Bowker, Les Gasser, Leigh Star and
William Turner (Erlbaum, 1997) pages 27-56. A theory of information based on
social interaction; it provides the philosophical foundations of our approach
to semiotics. A gzipped postscript version is
also available.
- On Notation, by Joseph Goguen. Some
basics of Peircean semiotics with easy applications to computer science and
mathematics. Revised version of a paper in TOOLS 10: Technology of
Object-Oriented Languages and Systems, edited by Boris Magnusson, Bertrand
Meyer and Jean-Francois Perrot (Prentice-Hall, 1993) pages 5-10.
- Social and Semiotic Analyses for Theorem
Prover User Interface Design, by Joseph Goguen, Formal Aspects of
Computing, volume 11, pages 272-301, 1999. A systematic justification of
the style guidelines for the proof websites generated by the Kumo system, based on algebraic semiotics,
narratology, cognitive science, etc.
- Signs and Representations: Semiotics
for User Interface Design, by Grant Malcolm and Joseph Goguen, in
Visual Representations and Interpretations, edited by Ray Paton and
Irene Nielson, Springer Wrokshops in Computing, 1998 (proceedings of a
workshop held in Liverpool), pages 163-172. An informal introduction to
algebraic semiotics with examples, including aspects of operating systems
interfaces.
- Algebraic Semiotics, ProofWebs and
Distributed Cooperative Proving, by Joseph Goguen, Akira Mori and Kai
Lin. This is an older description of the tatami ProofWeb data structure and
the Kumo proof assistant and website generator, emphasizing how semiotics was
used in their design. In Proceedings, User Interfaces for Theorem
Provers (Sophia Antipolis, France, 1-2 Sept. 1997), pages 25-34.
See also the Laboratory for Comparative
Human Cognition homepage.
Algebraic
Semiotics
Semiotics is the study of signs. Our research attempts to make this
area more systematic, rigorous, and applicable, as well as to do justice to
its social and cognitive foundations. Algebraic semiotics
combines aspects of algebraic specification and social semiotics. It has
been applied to information visualization, user interface design, the
representation of mathematical proofs, multimedia narrative, virtual
worlds, and metaphor generation, among other things. The course CSE 271 includes a relatively non-technical
exposition of algebraic semiotic and many of its applications. More
detailed technical information may be found in
Semiotic Morphisms, Representations, and Blending for User Interface
Design, Foundations for Active
Multimedia Narrative: Semiotic spaces and structural blending, and
An Introduction to Algebraic Semiotics, with
Applications to User Interface Design. Results of this research
were used in building the Kumo theorem
proving system, along with ideas from narratology
(the systematic study of narrative).
A basic concept is that of a semiotic morphism, which provides
representations in one sign system (the target) for signs from another (the
source). Semiotic morphisms can be partial, i.e., they do not necessarily
have to preserve all of the signs or all the structure of the source
system. The degree to which semiotic morphisms preserve various features
provides a basis for comparing the quality of representations, and
leads to an interesting study of trade-offs. Our most recent work places
more emphasis on the direct integration of values with the entire design
process.
To illustrate these ideas, consider proofs (in some fixed logical system)
as forming a sign system; an extension of this system includes additional
information to help with understanding proofs, such as motivation, background
tutorials, and examples. Another sign system is given by website technology
(XHTML, JavaScript, XML, etc.). Then representations of proofs as websites are
morphisms from the first system (or its extension) to the second, and the
orderings on semiotic morphisms compare aspects of the quality of such
representations. Some website design principles, called the Tatami conventions, were extracted
from our study and embodied in our Kumo
tool, which combines proof assistant and website generation capabilities;
it generates so-called "proofweb" data structures that use HTML, JavaScript,
etc., which can then be viewed with any browser. For more details, see Web-based Support for Cooperative Software
Engineering. Some other applications are discussed in Information Visualization and Semiotic Morphisms
and in Steps towards a Design Theory for
Virtual Worlds.
The "world famous" UC San Diego Semiotic Zoo contains a collection of semiotic
morphisms, each an example of bad design arising through failure to preserve
some relevant structure. (Notes: (1) The zoo still has one wing under
construction; and (2) it won a "Creativity Award" from Art &
Technology.)
Mathematical foundations is provided by the rather recent and very
abstract field called "category theory" (it is not related to the area of
psychology of the same name), by noting that sign systems together with
semiotic morphisms form a category. Some modest additional axioms are
satisfied, which leads to the notion of a 3/2-category. An
appropriate notion of colimit for such categories has properties
that make it suitable for studying the blending of sign systems,
as explained in An Introduction to Algebraic
Semiotics, with Applications to User Interface Design. See also
Semiotic Morphisms, Representations, and
Blending for User Interface Design, to see how hidden algebra
extends algebraic semiotics to handle interaction. The papers Foundations for Active Multimedia Narrative:
Semiotic spaces and structural blending and Information Visualization and Semiotic
Morphisms include more intuitive introductions to many issues, and
the webnote Semiotic
Morphisms may be convenient. Much of our recent research on art and music and on qualia
and consciousness is also related, as well as our work on blending and
cognitive semantics.
Courses
Other Links
- The Distributed Cognition and HCI
Lab, in the UCSD Cognitive
Science Department, led by Professors Jim Hollan and Ed Hutchins.
- The Visualization Lab at the San Diego Supercomputer Center, led by Mike Bailey
and others.
- Website on
blending, with applications to metaphor; work of Gilles
Fauconnier, Mark Turner and others, in the area of cognitive linguistics. (An Introduction to Algebraic Semiotics, with
Applications to User Interface Design gives a mathematical
formalization of blending.)
- Best website I know on graphical design for the web, the Yale Style
Manuual.
- Homepage of Mark
Ackerman, at the University of Michigan; great stuff on CSCW, electronic
media, etc.
- A collection of proof displays
generated by the latest Kumo proof
assistant and website generator, version 4. This is part of the Tatami project, a goal of which is to make
machine proofs much more readable than is usual; the project has some emphasis
on behavioral proofs of distributed concurrent systems.
- Homepage of
Interactions magazine, published by the ACM.
- The Interface Hall of
Fame and Interface Hall of
Shame.
- Homepage of Ben
Shneiderman.
- Essays of Don Norman.
- The UCSD Semiotic Zoo.
- Homepage of Jennifer
Preece; see in particular, the subsite on her new book, Online Communities.
Maintained by Joseph Goguen
To the research projects index page
Last modified: Thu Dec 5 10:10:08 PST 2002