«Creativity Support for Computational Literature By Daniel C. Howe A dissertation submitted in partial fulfillment of the requirements for the degree ...»
Prior to reading virtual muse, playing with different digital art projects of my own, and observing the different art galleries and products of digital artists, I had not thought much of using programming to express the artistic creative side of me. I had built up a mental barrier between these two worlds that this class has broken down. I think that this class is just a first step in my personal exploration of art and technology as a tool of expression.
I just want to feel that what I’m working on, and the field I’m working in, are vibrant, changing, full of possibilities. That to sit down and write can mean and result in a lot of different things. That feeling is one I’ve had in this class (from the Markov models, from Hartman’s pragmatism, from the grammars) and that experience is one I’m grateful for.
If i [sic] have learned one thing, it is that code, like any other medium, can be coaxed, teased, and reworked into all kinds of forms, creating all kinds of subtle and expressive changes in a work.
the fact that this class focused on language-based projects more than visual art was also a somewhat surprising but ultimately really fascinating turn. For one, I am personally profoundly enraptured by the kinds of texts that generative grammars in particular make; but the way n-grams and other statistic- and probability- based algorithms play with language I found unexpectedly interesting. I would love to do some work with these kinds of algorithms that would combine creative production and textual analysis; in even our mini-projects I was enchanted by the idea that recombining poetic texts might not only produces a new creative work but also allow for the gleaning of new meaning and making new connections...
Additionally, the reception of this work by the larger digital art community suggests a degree of utility in these outputs which appears to have been facilitated, at least in part, by the tools employed. As renowned digital media theorist and practitioner John Cayley  writes about the student project gallery, Here's a gallery of proof that if you give expressive programming artists the right tools - Rita's programmable writing tools - then language will also drive their art, and make a literal art. As for literary artists, Rita gives them an articulate introduction to software art, and does so in a way that respects the language-making at the root of writerly practices. So many new things with novel aesthetics...
To further support the claim of value, we can note that projects generated in the class achieved significant success in external venues, including publication in prestigious digital literary journals, presentations at conferences, international awards, and grants for further work105. However, if we accept the general definition of creativity as outlined above, we must evaluate whether so-called “novel” outputs are generated by users of these tools as well. As a first step toward answering this question, we analyzed students’ final projects from the semester, looking specifically at the diversity of their output as an experimental indicator of novelty.
220.127.116.11 Evaluating “Novelty” “When evaluating the use of creativity support tools, we consider diversity of outcomes as an indicator of success. If the creations are all similar to one another, we feel that something has gone wrong.” [Resnick et al. 2005] In attempting to analyze the novelty of final project results, we looked at the diversity of student project outcomes, first considering the categories of media that the projects engaged. Our intuition here, supported by a number of researchers [Resnick 2000; Resnick et al. 2005] was that a outcomes created via support tools that do not adequately support novelty would group together into one or a few categories of output. The following categories were proposed for the analysis: Audio, Video, Text, Image, Animation, Text, Text-To-Speech, Performance, Installation, Interactive, Locative or Psycho-Geographic, Physical Details of these are withheld in order to preserve student privacy in accordance with UCAIS (human subjects) guidelines.
Computation, Networked, and Web-Art106, from which multiple codings were allowed. As shown in Table 11, projects spanned all of the aforementioned categories, with only three— Text (60%), Interactive (73%), and Web-Art (67%)—containing a majority of students. The popularity of these three categories was somewhat expected as nearly all examples and assignments during the semester fit into all of these categories. Audio (60%) was also a highly populated category, though an explanation of this result is less apparent. The topic of audio was not a central part of the course and only a few of the RiTa examples featured audio. Hypotheses for this finding include a) the fact that audio can generally be added to an existing project without much additional difficulty, b) several students had prior backgrounds in audio processing, and c) several examples works discussed in class contained audio components. Further study would be required to determine if this result is either significant or indicative of students’ affinity for aural media.
While a ‘Web-Art’ coding refers only to the project’s being accessible (and executable) on the web, projects coded as ‘Networked’ employ custom client-server networking as an integral component of the work.
Table 11: Final project attributes.
A second element of our analysis of diversity projects looked at the RiTa modules employed in each final project. Our intuition here was that a outcomes created via a support tool that was too narrowly focused (i.e., lacking “wide walls”) to adequately support novelty would leverage just one or a few modes of use. Here we coded each project with the specific RiTa modules that it employed (again multiple codings were allowed as many projects employed multiple parts of the toolkit.) Rather than proceeding by hand, codings here were produced automatically via source code examination (parsing both import statements and variable declarations) to minimize concerns of inter-coder reliability. Such analysis was enabled by the fact that students’ source-code was required among the final project deliverables. Several students (20%) did not use RiTa at all, and were thus coded in the single category ‘Did not use RiTa’.
In Table 11 we see that with the exception of Text-Display (53%), each of the modules were used by (at most) one-third of students. Several modules were either (unexpectedly) unused or used by less than 10% of students. These include the Web-Mining (0%), RiTa-Server (0%), WordNet (0%) and Audio (7%) components. Our hypothesis here is that each of these modules, with the possible exception of WordNet, are designed primarily to facilitate rapid development, a central design constraint for the toolkit (Design Constraints in Chapter 2). As such, they can often be substituted out in a final project by embedding their functionality elsewhere. The RiTa Server (also presented in Chapter 2) presents a case in point, as the module was designed primarily to enable micro-iteration in cases where loading times for large data models could be prohibitive, rather than to be used in ‘finished’ pieces.
Similarly, the RiSearcher object provides real-time n-grams (for small n) via the Google Search engine and can often be replaced by a more-efficient embedded n-gram model (that does not require network access) in cases where the lexicon can be pre-determined.
Table 12. RiTa modules used in final projects.
The findings in Table 12 suggest that significant variation was in fact present in student projects. And while a number of factors may have contributed to this finding, it seems likely that the affordances of the RiTa tools played some not insignificant part. Further evidence in support of this finding was the fact that the two mini-projects previously assigned focused heavily on the grammar and n-gram modules. Thus one might reasonably expect these modules would be used more far more frequently in student projects, but this was not the case (average of all modules=23%, n-grams=20%, grammars=33%). This finding also suggests that, as students gain facility with the tools and, concurrently, with digital arts and literary practice, they make progress toward the more general goal of procedural literacy. As literacy improves, their reliance on a single module, library, or even programming language, diminishes so that by the end of the semester students are capable of navigating APIs, documentation, and source-code, using tools that were not specifically discussed in class, and ignoring elements of tools designed to provide scaffolding for specific tasks which are no longer perceived as problematic. As discussed in Chapter 3 (Pedagogy), well-designed scaffolding provides support only for as long as the skill for which it is designed remains difficult. When this is no longer the case, the scaffolding should become not only unnecessary, but more importantly, transparent—incrementally illuminating the conceptual details it once allowed the learner to avoid in lieu of more pressing concerns.
5.8 Limitations The further one moves away from the controlled laboratory situation the more difficult it becomes to establish a clear [and] unambiguous set of relationships that support valid conclusions. Research based in actual practice often supports many alternative explanations of what happens and how it happens. [Hewett et al. 2005] There are number of limitations to the pilot evaluation presented, specifically the lack of a control group, the nature of our participant pool, and the presence of a highly motivated experimenter. Further, as discussed in Denny , Parsons’ problems exhibit a number of limitations unrelated to this study.
5.8.1 Lack of Controls As this pilot study is contextual, it exhibits both the strengths and drawbacks of this methodology. One such drawback in this case is the lack of any formal control group; that is, one or more sections of the course in which some other toolset and pedagogical approach was used. Unfortunately this type of experiment has thus far been infeasible in the context of PDAL. As such, two important questions remain unanswered as regards the efficacy of our tools and teaching strategies. First it is unclear whether one factor or multiple factors in combination are responsible for the results presented above. As noted in Hewett et al , a weakness of conceptual methodologies is the fact that, the further one moves away from the controlled laboratory situation the more difficult it becomes to establish a clear unambiguous set of relationships that support valid conclusions.
Thus, in our study, students’ gains in self-efficacy and performance may well be due to a number of factors: the selected tools, the pedagogical philosophy, the assignments and exercises, the public critiques—all of which are rather unique in computer science education—or some complex combination of one or more of these. Additionally, it is unclear to what degree students’ measured progress was due to our approach or to other (external) factors; e.g., outside exposure to programming, or learning accomplished in other classes. As
Turner  states:
It is difficult to precisely tell the effect of the presence of a technology and methodology, since there is no way to observe a creative process then ‘rewind’ it, insert the new technology and methodology, press ‘play’ and compare the results. The difference must be subjective, at least at small and medium scales of deployment. By using the term ‘prefer’, I have chosen one factor that could be said to represent an improvement in the creative process—that the creative worker prefers it. Other questions I could have chosen are, for example, whether the tools or methodologies made the process ‘quicker’, ‘cheaper’, ‘more prize-winning’, and so on, but these are difficult to evaluate in the context of interactive art at small or medium scales.
5.8.2 Participants A further potential source of bias in the study was the fact that participants were from a narrow and non-representative demographic (specifically university and graduate students at two top-tier universities). Thus, it is unclear whether the effects noted would generalize to other demographic groups, still less so for under-represented groups [Plass et al. 2007].
Further, all participants in the study self-selected by enrolling in the course and consenting to participate in the study. As the course was not required, it is likely that participants had at least some prior motivation for learning the material in question, casting further doubt on the generalizability of our results.
5.8.2 Experimenter As is often the case in contextual studies of this kind, the course instructor and software creator was also the author of the study and clearly influenced participants’
experience of both the tools and teaching. As mentioned previously (See Chapter 3:
Generative Pedagogy), this helped to facilitate the adaptive character of the course, as pace and direction were adjusted based on feedback about both the tools and the materials. It is unclear whether a course taught by a different instructor, specifically one less familiar with these tools, would yield similar results.
5.8.2 Software Similarly, as is often the case in contextual studies of this kind, the software being evaluated was modified throughout the course of the study. Several bug fixes, and even a number of features requests were accommodated over the course of the semester, thus introducing another potentially confounding factor to the study.