Computational Narratology
Contents
Introduction

A joint project with Fox Harrell, entitled "Computational Narratology," is developing theory and tools for new generation interactive media, to enable integrated narratives, metaphors, images, etc. to be generated on the fly, in response to user input. Its main component is a novel algorithm called Alloy, to generate new structures by blending, based on recent research in cognitive linguistics, computer science, and semiotics; in particular, it uses the algebraic semiotics formalization of the cognitive linguistics theory of how metaphors are constructed, and more generally, how conceptual spaces are combined (see Foundations for Active Multimedia Narrative: Semiotic spaces and structural blending for details - the complete citation is given in the bibliography below). The semiotic spaces of algebraic semiotics are used, rather than the mental spaces that Fauconnier developed for cognitive linguistics, because we need the greater generality of n-ary relations, structure building functions, types, and axioms, in order to blend structures at the syntactic and discourse levels, as well as generate novel metaphors; we also need the greater rigor in order to design and build the computer algorithms. Our initial experiments used the blending algorithm in the Griot system, to produce polypoems, which are a new art form, consisting of a very large family of poems having a common theme and qualitative feeling, but with varying content and structure. Sample element of two such polypoems are given on the Griot system homepage. The first uses Labov's structural theory of narrative (see Notes on Narrative) at the discourse level, but there are many other possibilities, e.g., the haibun form made famous by Basho is used in the second example (for more details, see The Griot Sings Haibun.)

An important recent finding is that the optimality principles given in Chapter 16 of The Way We Think by Fauconnier and Turner (Basic 2002) for determining which blend should be chosen, do not work well for some applications, such as generating metaphors in the style of some late 20th century poetry, and for blending the parts of an interface in user interface design. For example, generating metaphors in the style of Pablo Neruda requires principles that are opposite to some of the common sense principles of Fauconnier and Turner (see Style as Choice of Blending Principles), while design applications require principles that take much more account of structure (see An Introduction to Algebraic Semiotics, with Applications to User Interface Design, the full citation for which is given below) and of dynamics, i.e., of change over time (see Semiotic Morphisms, Representations, and Blending for Interface Design, which uses hidden algebra for this). Because this is a formal theory with a computer implementation, its optimality principles are necessarily based on formal features such as degree of preservation of structure, degree of type mismatch, and degree of axiom preservation. We are exploring how much can be done within this limitation, and our initial assessment is that quite a lot is possible.

A recent conceptual advance is to unify conceptual spaces in the sense of Fauconnier (which are also called mental spaces) with conceptual spaces in the sense of Gardenfors. The former are logical in nature, while the latter are geometrical in nature, and a frame is a relation of "meaning" that holds between the two, indicating which "concepts" correspond to which "percepts". Frames are similar to the classifications of Barwise and Seligman and the formal contexts of Wille, but go further in allowing much more logical structure in concepts and much more geometrical structure in percepts. They are also implicitly parameterized by "signatures," which provide a notion of context, and this is made explicit by providing families of frames (which formally are functors from a category of signature to a category of frames - and hence are formally the same as institutions. Moreover, we argue that it is frames that should be blended, not just their logical or meta-data components. For technical details about frames, see Information Integration in Institutions.

Brief Bibliography
  1. [New] There is now a Griot System Homepage, with links to a performance that used Griot on 28 October, celebrating the opening of the California Institute for Tececommunications and Information Technology building, in cooperation with the UCSD Center for Research in Computing and the Arts.
     
  2. [Newly Revised] Information Integration in Institutions, for Jon Barwise memorial volume edited by Larry Moss, Indiana University Press. This paper unifies and/or generalizes several approaches to information, including the information flow theory of Barwise and Seligman, the formal conceptual analysis of Wille, the lattice of theories approach of Sowa, the categorical general systems theory of Goguen, and the cognitive semantic theories of Fauconnier, Turner, Gardenfors, and others. Its rigorous approach uses category theory to achieve independence from any particular choice of representation, and institutions to achieve independence from any particular choice of logic. Corelations, cocones, and colimits over arbitrary diagrams provide a very general formalization of information integration, and Grothendieck constructions extend this to several kinds of heterogeneity. Examples from databases, ontologies, cognitive semantics and other areas are treated. An unusual way to institutionalize databases is given in an appendix, inspired by C.S. Peirce's triadic semiotics. A postscript version is also available.
     
  3. Style as Choice of Blending Principles, by Joseph Goguen and Fox Harrell. In Style and Meaning in Language, Art, Music and Design, Proceedings of a Symposium at 2004 AAAI Fall Symposium Series, Technical Report FS-04-07, AAAI Press, 2004, pages 49 to 56 (Washington DC, October 21-24). There are also a postscript version, and powerpoint slides for the lecture. This paper proposes a new approach to style based on the principles for blending that works employ; it also includes an implementation approach to syntax based on structural blending and cognitive grammar, and proposes a reconsideration and generalization of optimality principles for blending. A poetry generation system based on this ideas is also explained, and some output is included. A longer version is in preparation for publication in a book of proceedings.
     
  4. Foundations for Active Multimedia Narrative: Semiotic spaces and structural blending, by Joseph Goguen and Fox Harrell. Under revision for publication in Interaction Studies: Social Behaviour and Communication in Biological and Artificial Systems.
     
  5. Formalization and Implementation for Cognitive Semantics, slides for keynote address at Workshop on the Potential of Cognitive Semantics for Ontologies, Torino Italy, 3 November January 2004. Sorry, it is in MS Powerpoint.
     
  6. Notes on Narrative, by Joseph Goguen. A brief webnote surveying some techniques for the analysis of stories, including summaries of the structural theory of narrative, and techniques for the extraction of value systems from stories.
     
  7. Time, Structure and Emotion in Music, by Joseph Goguen and Ryoko Goguen, to appear in book of University Lectures at Keio University in 2003-2004 academic year; there is also a postscript version. Describes experiments with music as stimulus that reveal unexpected properties of the chunking of qualitative experience, and sketches a theory of qualia based on the blending model of algebraic semiotics.
     
  8. Steps towards a Design Theory for Virtual Worlds, by Joseph Goguen. Chapter in Developing Future Interactive Systems, edited by Maribel Sanchez-Segura, published by Idea Group, pages 116-152, 2005; an html version is also available. This paper sketches algebraic semiotics and its applications, especially to user interface design and scientific visualization
     
  9. Semiotic Morphisms, Representations, and Blending for User Interface Design, by Joseph Goguen, in Proceedings of AMAST Workshop on Algebraic Methods in Language Processing (Verona, Italy, 25 - 27 August 2003), pages 1-15. This paper extends algebraic semiotics to handle interaction by using shows how hidden algebra; some examples are given in detail. A pdf version of the paper is also available.
     
  10. Musical Qualia, Context, Time, and Emotion, in Journal of Consciousness Studies 11, 3/4, pages 117-147, 2004; Art and the Brain, Part III . This is a paper on the philosophy and cognitive science of music, using Husserl's phenomenology of time consciousness, hierarchical information theory, dynamical systems theory, and blending theory. A pdf version is also available.
     
  11. [New] Rivers of Conscousness: The Nonlinear Dynamics of Free Jazz, by David Borgo and Joseph Goguen, to appear in Proceedings, Annual Meeting of International Association of Jazz Educators, Long Beach CA, 5 to 8 January 2004. Sorry, it is in MS Word.
     
  12. Information Visualization and Semiotic Morphisms, by Joseph Goguen and Fox Harrell. An informal introduction to the notion of semiotic morphism from the field of algebraic semiotics, showing how information visualization, in both analysis and design, can benefit from a viewpoint based on structure-preserving morphisms. In Multidisciplinary Approaches to Visual Representations and Interpretations, ed. Grant Malcolm (Elsevier 2004), pages 93-106. This book consists of revised versions of papers from the Second International Conference on Visual Representations and Interpretations, Liverpool UK, 9-12 September 2002. An older html version of the paper is also available. Background for this paper can be found in the webnote Semiotic Morphisms and the paper On Notation.
     
  13. Extended abstract of Sync or Swarm: Group Dynamics in Musical Free Improvisation, by David Borgo and Joseph Goguen, and shorter abstract, in Proceedings, Conference on Interdisciplinary Musicology, Dept. Musicology, University of Graz, 2004; the short abstract appears on pages 52-53, and the extended abstract in the attached CD. Held in Graz, Austria, 15-18 April 2004. Sorry, both are in MS Word.
     
  14. An Introduction to Algebraic Semiotics, with Applications to User Interface Design, by Joseph Goguen, in Computation for Metaphor, Analogy and Agents, edited by Chrystopher Nehaniv, Springer Lecture Notes in Artificial Intelligence, volume 1562, 1999, pages 242-291. This is the original paper on the mathematical foundations of algebraic semiotics, with 3/2-categories and 3/2-colimits, which provide a foundation for blending with any optimality principle given as a partial ordering on morphisms; the paper also has many examples, especially from user interface design.
Some Other Links
Maintained by Joseph Goguen
To the projects index page
Last modified: Sun Jan 1 09:09:34 PST 2006