«UNIVERSITY OF CALIFORNIA, SAN DIEGO Theory and Technology for Computational Narrative: An Approach to Generative and Interactive Narrative with Bases ...»
UNIVERSITY OF CALIFORNIA, SAN DIEGO
Theory and Technology for Computational Narrative: An Approach to Generative and
Interactive Narrative with Bases in Algebraic Semiotics and Cognitive Linguistics
A dissertation submitted in partial satisfaction of the
requirements for the degree Doctor of Philosophy
Computer Science and Cognitive Science
Douglas Alan Harrell, Jr.
Committee in charge:
Professor Geoffrey Voelker, Chair Professor Gilles Fauconnier, Co-chair Professor Lev Manovich Professor Simon Penny Professor Victor Vianu Copyright Douglas Alan Harrell, Jr.
(a.k.a. D. Fox Harrell) All rights reserved.
OF THE DISSERTATIONTheory and Technology for Computational Narrative: An Approach to Generative and Interactive Narrative with Bases in Algebraic Semiotics and Cognitive Linguistics by Douglas Alan Harrell, Jr.
Doctor of Philosophy in Computer Science and Cognitive Science University of California, San Diego, 2007 Professor Geoffrey M. Voelker, Chair Professor Gilles Fauconnier, Co-Chair This dissertation presents theoretical and technical support for, and implementations of, narrative computational media works with the following characteristics: generative content, semantics-based interaction, reconfigurable narrative structure, and strong cognitive and socio-cultural grounding.
A system that can dynamically compose media elements (such as procedural computer graphics, digital video, or text) to result in new media elements can be said to generate content. The GRIOT system, a result of this dissertation, provides an example of this. It has been used to implement computational poetry that generates new narrative poems with varying particular concepts, but fixed themes, upon each execution. This generativity is enabled by the Alloy system, which implements an algorithm that models key aspects of Gilles Fauconnier and Mark Turner’s theory of xiv conceptual blending. Alloy is the first implementation of Joseph Goguen’s algebraic semiotics approach to blending. (Fauconnier & Turner, 2002; Goguen, 1998) This research also contributes to the theory of algebraic semiotics by developing a blending-based notion of style.
Semantics-based interaction means here that (1) media elements are structured according to the meaning of their content, and (2) user interaction can affect content of a computational narrative in a way that produces new meanings that are constrained by the system’s author. “Meaning” in this case indicates that the author has provided formal descriptions of domains and concepts pertinent to the media elements and subjective authorial intent.
Meaning can also be reconfigured at the level of narrative discourse. The formal structure of a computational narrative can be dynamically restructured, either according to user interaction, or upon execution of the system as in the case of narrative generation.
Strong cognitive and socio-cultural grounding here implies that meaning is considered to be contextual, dynamic, and embodied. The formalizations used derive from, and respect, cognitive linguistics theories with such notions of meaning.
Furthermore, the notion of narrative here is not biased toward one particular cultural model. Using semantically based media elements as a foundation, a cultural producer can implement a range of culturally specific or experimental narrative structures.
1.1 Goals and Perspectives Imagination, computation, and expression blend in the goal of inventing new narrative forms for new media. New narrative forms may become lost to history as mere curiosities, or they may revolutionize and invigorate our possibilities for communicating our thoughts, feelings, dreams, social constructions, and even our senses of selves to one another. New media technologies, likewise, may fade to grey obsolescence, or they may become the basis for new conventions sending its artists and theorists scrambling to discover its language – a language that allows one to manipulate the characteristics of the medium to convey a range of expressive work ranging from functional data to evocative fictions. The hope here is to offer theory and technology as a step on the path toward the revolutionary and conventionestablishing fates, while leaving behind the disappointments of the lost and obsolete.
The basis for this hope is a novel and integrated perspective on creative imagination, computer science, and cultural expression.
While this dissertation offers specific advances in computational narrative, perhaps its central contributions are theoretical and methodological. Unified by the overarching goal of expression and a central concern for narrative, this research tightly integrates approaches from computer science, cognitive science, and artistic
production such that each of these areas mutually informs the others:
1) Imaginative Cognition | Computer Science Computer science allows for formalization, precise modeling, implementation, and experimentation with recent cognitive science theories that understand cognition as distributed, embodied, and
2) Computer Science | Artistic Production Artistic production drives computational solutions and enables critical technical practices by introducing subjectivity and human
3) Artistic production | Imaginative Cognition Cognitive science investigation of imagination provides scientific accounts of how narrative imagining, metaphor, conceptual integration, and related phenomena are fundamental to human thought, and of how imaginative fiction maps on to real human
The goals and approaches outlined above are important because I believe that computing can contribute to understanding and expressing nuances of the human social condition, a domain traditionally investigated by the humanities. Narrative is central to human communication. However, many forms of computer-based interactive narrative have failed so far to capture the popular consciousness.
Computer gaming has produced many examples of popular narrative experiences, and both computer gaming artifacts and theory of computer gaming deeply inform the work here. However, many computer games are not primarily narrative, and even those that are often do not feature technical support for allowing user interaction to dynamically generate narrative meaning (though players undoubtedly generate their own narrative interpretations of gameplay). I believe that an approach that takes seriously culturally based forms of narrative imagination and its expression, at the same time as considering the (cognitive) scientific underpinnings of that imagination and its expression, and looking critically at the ways that computational media can allow for manipulation of those statements at the semantic level, holds great promise for the development of effective computational narratives.
This dissertation does not define a singular eventual form of computational narrative such as virtual reality, hypertext, or narrative computer games. Instead, the starting point is meaning and narrative cognition. I propose a model to allow an author to represent her or his subjective meanings, and I theorize how these meanings can interact with knowledge bases of media assets, data structures for representing a wide range of discourse structures, and an engine to orchestrate interactive events.
This basis allows the author of an interactive narrative to utilize a combination of semantic data structures, algorithmic blending of those structures, a reconfigurable model of narrative structure, and a subjective (but less easily manipulable by computational means because semantics are not explicitly represented) knowledge base. My approach starts by considering what meaningful content an author has to express, and how structure and interaction can enable expression of that content.
This approach encourages an author to utilize whatever computational means necessary in order to convey her or his particular narrative. At the same time, it also allows an author to create constructive spaces in which user interaction or content production are the core expressive aspects of the work. This point of view contrasts to those that start from imagining one futuristic form of computational narrative as a “holy grail,” for example, allowing users to manipulate objects and interact with characters within a narrative virtual world. The theoretical bases proposed here are sufficiently general as to be used to articulate models of immersive virtual worlds, computationally generative poetry, narrative computer games, and other related forms.
The view of narrative and poetry taken in this dissertation can be captured by
integrating the following quotations:
Poetics deals with problems of verbal structure, just as the analysis of painting is concerned with pictorial structure. Since linguistics is the global science of verbal structure, poetics may be regarded as an integral part of linguistics. (Jakobson, 1960) … models of language and linguistic organization proposed should reflect what is known about the human mind, rather than purely aesthetic dictates such as the use of particular kinds of formalisms or economy of representation … (Evans, Bergen, & Zinken, 2006) Poetry, narrative, and their analyses are seen as within the domain of inquiry of linguistics (if not the structuralist model of Jakobson’s time). Linguistic phenomena are seen as observable manifestations of human cognitive processes. (Fauconnier,
2000) Since I am interested in developing new media forms and genres that may or may not possess all of the characteristics of narrative or poetry from strictly literary traditions, my view is necessarily broad. I begin with cognitive perspectives on narrative imagining, and its component structures and processes, as basic semantic “building blocks.” Poetry is taken to be the wide domain of verbal art, which is not given status greater or less than other linguistic phenomena. Indeed, a major insight of the cognitive linguistics enterprises is that the same cognitive processes involved in everyday common sense reasoning also underlie literary creativity.
This does not mean that I ignore the cultural specificity of narrative and poetic forms or the entire history and insight to be gained by literary theoretic or other approaches to narrative. Rather, I invoke cognitive definitions in order to admit a wide range of specific influences and insights. The following is intended to roughly distinguish this approach from related approaches in the area of narratology (the study of narrative). This discussion is meant to provide an orientation for readers unfamiliar with this field.
There is no consensus on definitions of terms such as “story” or “narrative.”
N. Katherine Hayles summarizes the situations well in (Hayles, 2005) as follows:
The binary established by the Russian formalists of fabula and sjuzhet followed the distinction, dating back to what Gerard Genette calls the “pre-history” of narratology, of story and plot. Mieke Bal defines fabula as the “material or content that is worked into a story,” while the story itself is “defined as a series of events.” This definition is more or less echoed by Genette, Seymour Chatman, Shlomith Rimmon-Kenan, and others. The sjuzhet, on the other hand, is the order of appearance of the events in the work itself, or [as] Chatman, quoting Boris Tomaskevsky, puts it, “‘how the reader becomes aware of what happened.’” Different theorists transpose these older terms into binaries with slightly different inflections, including story and discourse (Chatman), fabula and story (Bal), and story and narrative (Genette), who sees both these terms deriving from a third term, narrating. As these examples show, there is no consistent terminology… Within cognitive science, Mark Turner has described stories as “dynamic interactions of events, actors, and objects,” presenting a quite minimal model of narrative aimed at capturing the skeletal pattern underlying narrative imagining.
(Turner, 1996) This does seem to parallel the minimal literary theoretic definition provided by Manfred Jahn in (Jahn, 2005), in which stories are sequence of events involving characters (‘events’ including “both natural and nonnatural happenings”).
Jahn also presents a minimal definition of narrative as “story presented via media,” as in Genette’s terminology in the Hayles quotation above. As a broad frame, this dissertation shall accept Turner’s view of narrative, with the realization, however, that it is quite a general description narrative when analyzing the rich conventions, innovations, and meanings of any specific cultural form. Under such view of narrative, the examples of computational poetry discussed here are also considered to be narrative, though the underlying technology is not limited to producing narrative discourse.
It is informative to distinguish how this cognitive linguistics perspective on narrative affects the framework for analysis. Jahn describes two major approaches to narrative analysis as “discourse narratology,” which “analyzes the stylistic choices that determine the form or realization of a narrative text,” and “story narratology,” which “focuses on the action units that ‘emplot’ and arrange a stream of events into a trajectory of themes, motives and plot lines.” (Jahn, 2005) David Herman describes another approach called “cognitive narratology,” in which “both narrative theory and linguistics should instead be construed as resources for cognitive science. Or rather, narratology, like linguistics, can be recharacterized as a subdomain of cognitivescientific research. From this perspective, both language generally and narrative specifically can be viewed as tool-systems for building mental models of the world.” (Herman, 2000) From the perspective of cognitive linguistics, cognitive narratology would focus on elaborating, for example, projection of action-stories onto eventstories, how these arise stories arise from image schema, conceptual metaphor and blending, and related cognitive phenomena described later in Section 2.2.