«Using a social semiotic approach to multimodality: researching learning in schools, museums and hospitals Jeff Bezemer, Sophia Diamantopoulou, Carey ...»
In taking a drawing as a sign of learning we suggest that as a result of the process of engagement with a part of the world, the child has made meanings for himself, outwardly, visibly, new ways of conceiving of the world, new ‘concepts’, and has integrated these into his inner conceptual resources. In that process the child’s entire set of resources is transformed, the resources have been augmented: learning has taken place. The child has achieved an augmentation of his capacities for representation, through his making of new meanings. In that approach, every drawing, any representational form, every sign made, is new, an innovation; its making is ‘creative’. The serial, ongoing process of transformative engagement, integration and inner transformation, together with the newly resultant state, constitutes learning. This is so whether the sign has been made outwardly – as in the drawing or in something said or gestured – or inwardly, in the process of engagement, selection, transformation, ‘inwardly’. Whether in meaning-making or in learning, interest is decisive. It forms the basis of the choice of what is taken as criterial about the entity for representation (the wheels of a car); of the apt means for representation (e.g. a drawing instead of speech); and for transforming that with which the learner has engaged.
NCRM Working paper 01/12
These concepts –signs of learning and transformative engagement- can be used to analyze drawings in any context. For instance, in the museum study introduced above the researchers tried to gain insight in visitor experiences by asking them to ‘remake’ the meaning of the exhibition as a drawing (instead of asking them to produce a spoken narrative). Figures 5-6 show the drawings of two visitors. Figure 5 shows the drawing of a 12 year old boy visiting the museum with his mother.
He chose to construct a drawing that shows an aeroplane, a tree, a spear, a tool and a skull.
These elements ‘stand for’ items that were particularly salient for this visitor. His attention had been drawn – among other things - by a small model of an aeroplane within a diorama, which explained that the contemporary site of Heathrow airport was a site of archaeological importance.
The drawing shows his interest starkly. His experience of the exhibition was in a significant way shaped around the model of the aeroplane and what that evoked for him, in the environment of that exhibition.
Figure 5: A twelve year old boy’s drawing
In another instance, Figure 6, the drawing of an 18 year old woman shows the skull of a bull at the top left of the map while the rest represents scenes from everyday life in a settlement. The main function of the skull may be to indicate the presence of just such a large skull at the entrance to the exhibition, quite clearly the model for the one she drew as a key feature. Her drawing transforms the resources she had selected in a far-reaching fashion, and re-frames them to create an account to explain the skull. In a multimodal social semiotic approach this counts as signs of what she has ‘learned’, for instance, about the tools people used for hunting and cutting, how they made pots for cooking and how they prepared their food.
Figure 6: An 18 year old visitor’s drawing More specifically, the examples show what selections and alterations the visitors made as they represented the exhibition as a drawing. From a multimodal social semiotic perspective these selections and alterations point to the (socially shaped) interests of the drawers.
Similar processes of transformative engagement can be observed in classrooms. Figure 7 shows some of the drawings made by primary school students in response to the teacher’s demonstration of forces described above. Without exception the class drew the bar magnets displayed on the screen as rectangles in a horizontal alignment. In this way, they demonstrated their learning about the experimental conditions that would be required in subsequent hands-on investigation. Not mentioned in speech, this interpretation was construed from their teacher’s positioning of the objects. At this point in the lesson, no reference had been made to the two poles of a bar magnet.
The students had to decide whether or not the differently coloured ends were significant for the experiment. Positioning as separation and conjoining was a dominant means of showing predicted experimental outcomes.
Figure 7: ‘Magnets’: Dry-wipe whiteboard hypotheses
In the context of the lesson, a drawing alone was not only an entirely valid response to the teacher’s instruction, but was perfectly adequate where everyone shared the knowledge that divided rectangles represented bar magnets and that their positioning hypothesized the result of a forthcoming investigation. This ‘situated obviousness’ may not be sustained beyond what was ‘the shared known’ of this lesson. Some children added writing, either to lexicalize the drawn prediction or to distribute experimental conditions and outcomes between the modes of writing and drawing.
Four students added wavy lines between their images as a means of theorizing magnetic force.
So what were the signs of learning in these drawings? Firstly, the task demanded attention to the curricular subject matter of magnetic force within a particular pedagogic framing. Certain experimental conditions were stipulated, and prior experience and knowledge were brought to bear in the context of introductory activities and interactions. Imagining possibilities by showing or lexicalizing movement was sufficient for the time being.
NCRM Working paper 01/12
(81 per cent of the class predicted attraction, which is scientifically incorrect.) At this point ‘getting it right’ was not an imperative; by the end of the lesson, it was. Secondly, the task entailed remaking of the teacher’s demonstration. In interviews, students talked about their experience of magnetic attraction in earlier schooling and at home (e.g. games and fridge letters), but they had not made this sort of prediction before. In representing their hypotheses graphically, there was learning as they selected the resources of drawing and writing for what was, for them, a new purpose. Thirdly, the task of hypothesizing was just one activity amongst others. The students also discussed and enacted the processes of attraction and repulsion, and, in a summative worksheet, arrows were stipulated as a resource for showing directional movement. As far as the lesson was concerned, these whiteboard predictions were incremental in the process of learning.
3.2. Bodily action
So far we have identified and analyzed signs of learning in drawings. These drawings were produced at different times and places. The museum visitors made their drawings up to an hour after their first engagement with the exhibition, in a room away from the exhibition. The school children made their drawings minutes after the teacher had demonstrated what might happen when two magnets are pushed together, whilst still in the same classroom. Now we turn to analyzing signs of learning in body movements, produced as immediate responses to the movements of others. Indeed, a multimodal social semiotic perspective draws attention to ‘interaction’ as much as to the ‘artefacts’ produced in interaction, such as the drawings discussed above; and it looks at interaction not just from the perspective of the ‘educator’ but also from the perspective of the learner, which is what we aim to do in this section. Here we take our examples from a study on learning in the operating theatre, but signs of learning in bodily action can and have also been studied in classrooms, museums and other contexts.
The example is focused on the interaction between a surgeon and a medical student as they are performing a (relatively minor) operation under general anaesthesia. The medical student’s task is to hold retractors in place, allowing the surgeon to excise a small lump just above the patient’s navel. We identified three ways in which the trainer and trainee positioned retractors during the operation. First, the trainer positioned the retractor herself and then handed it over to the trainee (see Table 1, Example 1), or adjusted the positioning of a retractor already held by the trainee by placing her own hands over the trainee’s. Second, the trainer described where the retractor needs to be positioned, for instance by saying, “Slide that one in a bit more laterally”, while pointing to a retractor held by the trainee (see Table 1, Example 2). Third, the trainee positioned retractors himself, on his own initiative and (seemingly) without directions from the registrar (see Table 1, Example 3). Table 1 provides close-up video stills of the different approaches to positioning the retractors.
Table 1: Three different approaches to the positioning of retractors Table 1 shows how teacher and learner use a range of communicative resources to manage the positioning of retractors. In Example 1 trainer uses hand gestures to signify to the trainee where and how to hold the retractor. Her verbal comment (“Just hold it there”) is only meaningful in relation to those hand gestures.
NCRM Working paper 01/12
In Example 2 her use of speech (“ just slide that one in laterally”) becomes more precise, describing a movement and a direction, whilst a pointing gesture indicates which of the two retractors to use for making that movement. In Example 3 speech is not used at all. Here the trainee signals to the trainer, through self-initiated positioning of the retractor, that he feels he is capable of doing so. This is then followed by an evaluation of the trainer (“Yeah, brilliant”; “yeah perfect”).
The control over the positioning of the retractors shifted in the course of the 14 minutes of dissection. We identified 22 instances of (re)positioning. In the first half, control lay mostly with the trainer as she positions the retractor herself in 13 out of 14 cases. In the second half, control was mostly shared by the trainer and trainee, with the trainer telling and pointing out where to (re)position the retractor which is handled by the trainee. In the second half of the operation the trainee also (re)positioned the retractor himself for the first time, on two occasions.
These changes can be taken as signs of learning: At first the trainer acts as a model, suggesting to the trainee how, where and when she positions the retractors. The trainee then begins to position the retractors himself, demonstrating that he has ‘picked up’ what is required in this context. Thus we can see how ‘designs for’ and ‘signs of’ learning are intertwined in interaction, with the teacher giving prompts in different modes and the learner displaying responses to these prompts.
4. Historical comparisons of multimodal designs for learning So far we have analyzed multimodal designs for and signs of learning produced by individual educators and learners on a particular occasion. In this section the focus is on changes in the semiotic, social and technological landscape. Multimodal social semiotics often makes historical comparisons to investigate how new technologies reshape the resources available to meaning makers. For instance, early printing technologies limited the kind, placing and number of images in textbooks, yet now not only do there seem to be more images than before, they often seem to dominate the page. In a different set of media, the shift from the blackboard to the interactive whiteboard has led to an increase in the use of visual means for the presentation of science and other subjects. Concerns have been expressed about such changes in the semiotic landscape, about the increased use of images for instance, and their implications for learning. To some commentators this threatens literacy, must lead to a general ‘dumbing down’, and is bound to have deleterious effects on economic performance. Less prominent, if equally firmly expressed, are beliefs in the empowering potential of such changes by their offering new routes into existing curriculum topics. Multimodal social semiotics can contribute to this debate by identifying what might be lost but also what might be gained from changes in the semiotic landscape.
The examples are of secondary school classrooms in England, and the subject taught is English.
One example is from data collected in 2000 (Figure 8), and the other example is from the same classroom, involving the same teacher, again teaching poetry, but to a different class, recorded in 2006 (Figure 9). In the earlier lesson the teacher was using an Overhead Projector (OHP); in the more recent lesson she was using an Interactive Whiteboard (IWB). The curricular categories in both lessons were constant: ‘poetry and ‘persuasive language’. In the meantime, however, there had been significant changes in the landscape of school English with respect to the pedagogic organisation of the classroom and the ‘performance’ of the roles of the participants, the display of texts, and the process of textual analysis, each of which is described and discussed below.
Figure 8: An English classroom in 2000 Figure 9: An English classroom in 2006 A rhetoric of ‘democratisation’ is suggested by the contrast, over that period, in the display and function of student texts in the classroom. In the lesson from 2000, canonical English texts and teacher-made typed, laminated and framed texts were displayed on the front and sidewalls of the classroom, with some student texts displayed on the back wall. In 2005, student-made texts were digitally incorporated into the active pedagogic space of the classroom.