«Creativity Support for Computational Literature By Daniel C. Howe A dissertation submitted in partial fulfillment of the requirements for the degree ...»
5.8.2 Parsons problems While Parsons problems present a number of advantages over traditional code reading and writing test, there are also weaknesses to this approach. First, it is quite possible to solve a Parsons problem (even correctly) and still not understand at a deep level how the code works. Similarly, for weak students, the Parsons problem format provides an opportunity to guess an answer that is not available in code-writing questions, though it is less likely, as compared to multiple-choice style questions, that many of these guesses will be correct. Further, Parsons problem do not adequately address process, as they present only a single method, while nearly all real programs skip from method to method. Finally, in contrast to code-writing tests which show a high degree of correlation to Parsons scores, scores on code-tracing problems seem largely independent [Denny 2008].
5.9 Summary We have no delusions that evaluating tools is an easy task, but we also believe that the potential impact of improved tools would be enormous in amplifying and inspiring creativity. [Resnick et al. 2005] As noted above, there are significant issues with the largely descriptive evaluation presented here, perhaps most notably the lack of any true control, which is often cited as a weakness of contextual methods. Yet as an initial investigation into the real-world effects of these tools (and others like them) it is encouraging on a number of different dimensions.
Students’ attitudes, self-efficacy and ability all showed significant positive change measured between the pre- and post-semester surveys and programming quiz results. Additionally, though difficult to measure in a classroom environment, we see some evidence that the tools support student creativity. Students noted in increase in the frequency of their creative expression via computational media over the course of the semester and perceived their outputs to be of importance beyond the class. Further, their work was well-received by the larger digital arts community, with projects later receiving publication in journals of digital literature, arts grants, and international awards. Further, students’ projects exhibited surprising diversity both in terms of the types of media represented, and in the elements of the toolkit utilized. Of course, to verify any of the trends discussed here, additional research is needed, a topic discussed further in the concluding chapter.
6.1 Contributions This thesis has presented RiTa from a number of perspectives: as tools and affordances for practicing writers; as a pedagogical strategy, both for teaching procedural literacy to humanities students, and for engaging Computer Science students with creative practice; and as a real-world testing ground for creativity support principles, providing a unique context for assessing the efficacy of design and evaluation strategies. As such, the contributions of this research fall primarily into two related sub-fields: Creativity Support Tools (CST) research and Computer Science (CS) education, according to which they have been grouped below. In addition to contributions and future work in these areas we present several possible directions for additional evaluation of tools like those presented. Finally we conclude with a set of more speculative arguments based on our experience with artists and students as they engaged with these tools over the last several years.
6.1.1 Contributions: Creativity Support In the context of creativity support tools research, RiTa represents the first production-quality toolkit designed specifically for practicing computational literary artists, complete with thorough documentation, examples, and an extensive catalog of sample projects, each with source code, screenshots, and descriptive text. In addition to the implementation details of the toolkit, we have presented a series of extensions to traditional natural language algorithms developed specifically for the needs of computational writers.
Additionally we have created a number of auxiliary tools (e.g., the RiTaServer, the RiTaEclipse plugin, and the RiGrammarView application) to augment the library’s functionalities in several widely-used environments. Further, we have detailed the set of design criteria (and anti-criteria) that guided our implementation decisions and have enumerated the set of design tensions that arose in the course of our iterative development process. In detailing our resolutions to these conflicts, we have abstracted a set of principles which appear unique to creativity support in an arts context; specifically the need to support serendipity, inverted use, artistic misuse, and micro-iteration. Additionally, we have detailed the specific technical implementations with which we were able to realize the above principles, without sacrificing usability or performance goals. Most importantly we have distributed (and evaluated) these tools in real-world creative contexts, over the past three years, with a diverse population of users including students, practicing artists, and educators.
6.1.2 Contributions: Education In the context of digital arts and computer science education, RiTa represents the first end-to-end toolkit designed specifically to support courses in computational literature.
Additionally, the RiTa tools appear to accomplish several related but distinct pedagogical
to effectively engage computer science students with creative practice and introduce • new ways of thinking about the discipline;
to broaden interest in computer science for a diverse range of students, furthering • procedural literacy and computational thinking beyond the boundaries of the computer science department;
to positively affect students attitudes and beliefs about programming, as well as their • quantitatively-measured programming skills.
Finally, the RiTa tools, in combination with the pedagogical approach presented, and the Programming for Digital Arts and Literature (PDAL) course represent one of the first successful integrations of tools and pedagogy in a digital media context, for both undergrads and graduate students, focusing on language and literature. As such it represents a new, and potentially important context for educators and researchers with which to explore the viability of procedural literacy, expressive programming, and computational thinking for a broader demographic.
6.1.3 Artistic Strategies My most notable breakthrough, really, was when I realized my standard practice for CS classes doesn't work here. [PDAL student, 2009] It is evident from our experience, as described in the chapters above, that while there may be significant overlap between the strategies taught in a typical software engineering context, and thus designed into creativity support tools, some of these may be unproductive in an artistic and/or educational context. An obvious example is black-box style encapsulation, which, while generally a useful technique for API and/or library design, can often frustrate students’ efforts to understand the larger picture of how their software operates, and contradicts pedagogical theories that advocate transparency and/or self-directed, exploratory learning.
Perhaps more interesting still, there appear to exist artistic programming strategies which extend beyond, and may even contradict, those generally taught in introductory computer science courses. Examples presented in chapter three include strategies that leverage micro-iteration, artistic misuse, inverted use, and serendipity. The reverse also appears to be the case as techniques generally considered to be standard practice in many software development contexts can be perceived as counter-productive in an arts context. One such example is test-driven development which, at least during the early stages of a project, can significantly restrict the exploratory process so important to successful artistic outcomes.
These observations highlight two important conclusion for creativity support, both within and beyond the education environments we have been discussing. First, that context must be taken seriously at all levels of design; from workflow, to interface, to the design principles guiding even the most minute implementation details. This perspective corresponds to some recent research which demonstrates how social, political, and cultural assumptions and values can become embedded in technical artifacts at even the very lowest levels [Friedman et al. 1996, 2006; Flanagan et al. 2005b, 2007, 2008]. While the added effort required to analyze just what is at stake in such decisions may seem large at first, it pales in comparison to the time spent on software projects which fail, either partially or totally, due to their lack of attention to details such as these107.
Second, these observations highlight the need for educators to pay careful attention to the mindset of students and its relation to the context at hand. It has not been the case, at least in our experience, that skills learned in say... an introductory programming class, can be easily applied in another pedagogical context, or at least this should not be assumed to be the case by educators. As one student faced with this difficulty put it, “I feel like I need to...
really fit myself into the mindset of an artist right now. Given that this is the only computer science course I'm taking at the moment, one would assume that would be easy, but that's not true at all. I'm having a lot of difficulty shifting my mindset into a more appropriate one.” Or, in the words of another student, “My most notable breakthrough, really, was when I realized my standard practice for CS classes doesn't work here. I approached this class as a CS For an overview of Value-Sensitive Design and its applications in a number of different research and industry projects, see http://depts.washington.edu/vsdesign/projects.shtml.
programming class, which meant staying up late at the last minute and doing my work then.
This tactic doesn't lend itself to art in any shape or form, and though I am an artist, I haven't thought of computers in the same fashion.” Here again we see how, beyond principles and implementation strategies, careful attention to the specificities of the context in which computation occurs appears to be an important factor in the relative success of tools like these; a factor that appears still more relevant when the context is an educational one.
6.2 Future Research The research presented here on the RiTa toolkit and its use as a pedagogical tool suggests a number of different directions for further research in both the creativity support and educational communities. This section describes several additional projects in these disciplines that could be achieved with adequate institutional support and the right team of collaborators. They range in size from small, semester-long projects that could be accomplished by a single researcher (or even graduate student), to larger multi-year projects that might well require inter-departmental collaboration. The projects are organized according to the aspect of the research they would extend, whether the core toolkit itself, auxiliary support tools, one or more pedagogical applications, and evaluation measures, although several are applicable to more than one such area.
6.2.1 The RiTa Toolkit There are a number of potentially interesting ways in which the core RiTa toolkit could be enhanced via further research and development. Some ideas along these lines include more sophisticated generation strategies, speech recognition capabilities, support for additional graphics contexts, and support for mobile platforms.
Over the past few month we have performed some initial experiments toward the goal of adding more sophisticated generation support, specifically for the lexicalized Tree Adjoining Grammar (TAG) formalism [Joshi et al. 1975]. While the mildly-context sensitive properties of TAG grammars would provide a degree of expressive power beyond that of the basic context-free grammars implemented in RiTa, this functionality would need to be balanced with an API that minimizes the complexity associated with such techniques, perhaps along the lines of Stone’s  Taglet system. This would provide a further level to explore with students in discussions of regular expressions, state-machines and the language hierarchy. On the other hand, the callback functionality currently provided in the RiGrammar object surpasses the functional capabilities of a basic TAG grammar (that is, a TAG object could be simulated by the current RiGrammar implementation, but not vice versa.) With this in mind, any gains in pedagogical clarity, most relevant perhaps to those students interested in natural language research, would have to be weighed against the additional complexity and potential confusion that might result from the inclusion of multiple grammar objects.
rendering capabilities, similar to the way in which the Processing.js (http://processingjs.org/) library functions.
The inclusion of speech recognition capabilities in the toolkit is a feature requested by several students in the context of their PDAL projects. Toward this end, we have experimented with wrappers for the CMU Sphinx recognition framework which could enable users to define (and modify), in a plain text file similar to a RiTa grammar or addenda file, a context for speech recognition. Unfortunately, the data models required, at least in the Sphinx framework, are prohibitively large and would preclude execution in a web browser context.
This is not to say that such functionality is not worth exploring further, only that it would likely need to be added as an optional or supplementary module, rather than as part of the core RiTa tools.