«BY GEORGIOS DIAMANTOPOULOS A THESIS SUBMITTED TO THE UNIVERSITY OF BIRMINGHAM FOR THE DEGREE OF DOCTOR OF PHILOSOPHY DEPARTMENT OF ELECTRONIC, ...»
21 Table 1: Summary of past EAC research studies examined in this review. A study has shown partial support for the EAC model if any of its results are statistically significant and consistent with the EAC model. Further, a study is said to be unsupportive or to have shown no support for the model if none of its results are statistically significant or if none of its statistically significant results are consistent with the EAC model. Studies that related to the PRS are marked as “not relevant”.
This leads us to another inherent challenge in the direct examination of the EAC model:
validation of the subject’s cognition. While the representational system access required to answer a particular question can be linguistically pre-supposed (i.e. the access of a visual representation is required to recover purely visual information such as the colour of an object), how can one be truly certain that the subject has accessed the pre-supposed representational system to recover the information and nothing else without the use of neuroimaging technology such as functional magnetic resonance imaging (fMRI)? fMRI is a device which allows scientists to determine which part of the brain is active at any given point in time.
Farmer et al. (1985) provided subjects with real stimuli (pictures, tape-recorded sounds and textural objects) that they had to experience and later recall, presumably to guarantee that the representational system the subject used is the one intended. However, this is no different from pre-supposing that when the subject is asked to report on visual information, it is necessary for them to perform a visual access; the need for validation is still warranted. Take a textural object for example, such as a rock: even if it is supposed that the subject was blindfolded (no such mention by the authors), it is not necessary that the subject encoded only the kinaesthetic/textural aspect of the rock. Alternatively, the subject may form a mental visual image of what the rock may look like based on the kinaesthetic input – the “feel” of the rock.
Thus, this methodology has failed to warrant the type of representational system accessed or to further involve the phenomenological, subjective experience of the subject. The same approach is taken by Wertheim et al. (1986) and is thus subject to the same criticism.
In an attempt to deal with this fundamental issue of validating the subject’s cognition, some researchers collected accounts of the subject’s subjective experience (Cheney et al., 1982; Elich et al., 1985; Baddeley and Predebon, 1991); however, the methodology was not informed by psycho-phenomenology literature (see Mathison and Tosey, 2008a; Mathison and Tosey, 2008b).
For example, the method of introspective inquiry suggested by Beck and Beck (1984) in their critique of a related study (Thomason et al., 1980) is informed only by NLP literature. Imagery and introspection is an area of human psychology that has a long history of controversy (see e.g.
Horowitz, 1983) simply because of its own very nature. The information is retrieved from a subjective source, the person, and the question of the reliability of any gained information is very quickly raised (Mathison, 2006). NLP aims to study people’s subjective experience and might thus be expected to have an interest in the methods of psycho-phenomenology; if NLP is to be explored academically and any potential links to be established, it is imperative that any enquiries into NLP are informed by established methodologies such as psycho-phenomenology.
RECORDING, RATING AND SELECTING EYE-MOVEMENTS
One cannot dismiss the inherent difficulty in recording and rating the eye-movements without an
appropriate device such as an eye-tracker. To date, two different methodologies have been used:
a) real-time scoring by human observers; or b) video recording the eye-movements and scoring them later. Especially in the first case, the question of who does the rating is especially relevant;
the implicit assumption has been that eye-movements are easily discernable by (un)trained human observers in real-time or with the use of video-recording equipment.
The study of eye-movements has a long history and so does their measurement (Carpenter, 1988; Yarbus, 1967). The author hypothesises that direct-viewing was used in NLP studies because the EAC model is taught to be useful in real-time human interaction where the practitioner observes the eye-movements without technological aids. The use of inexperienced graduate student raters is advocated by Sharpley (1987) as traditional and a good measure of a procedure’s readiness and robustness. This argument could only be valid if the model’s suitability for adoption by untrained individuals was assessed and not its validity. The only available information regarding the accuracy and reliability of direct-viewing of eye-movements is that movements of less than 1° rotation (0.2mm movement of the retina) are not discernable by the naked eye (Yarbus, 1967; it is unclear whether this refers to a trained or untrained individual) and the question of sufficient reliability for the purposes of a scientific study quickly arises.
Indeed, several studies make no reference to the experience of the raters (Gumm et al., 1982;
Poffel and Cross, 1985; Farmer et al., 1985; Wertheim et al., 1986; Burke et al., 2003), while others have regarded the use of naive (Thomason et al., 1980; Cheney et al., 1982; Baddeley and Predebon, 1991) or briefly trained raters (Falzett, 1981; Ellickson, 1983; Elich et al., 1985;
Sandhu, 1991) as acceptable. NLP practitioners were used in one instance (Buckner and Reese 1987); again, this does not guarantee rating accuracy or reliability.
24 The case of some studies is strengthened because they used video-recording equipment that allows the rater to review eye-movements (Gumm et al., 1982; Cheney et al., 1982; Elich et al., 1985; Poffel and Cross, 1985; Wertheim et al., 1986; Sandhu, 1991; Baddeley and Predebon, 1991; Burke et al., 2003). In the methods used by most of these authors, the subject was forced to look at the camera thus restricting their head and body movement (Cheney et al., 1982). It is questionable whether all relevant eye-movements are discernable both because of relevant training and obscuring of the eye by blinks or head tilts and so on. The question of precision and reliability of rating has not been raised before other than inter-rater reliability tests which only certify a statistical agreement between raters and have no account for their individual abilities or other limitations imposed. In order to eliminate as many variables as possible, a recording and rating methodology whose error is known has to be used. In the methods described, no such precision/reliability tests have been performed. The question of what eye-movements occur during blinks and how they are relevant to the EAC model has not been considered in the literature other than by Buckner and Reese (1987) and Baddeley and Predebon (1991).
Even if an assumption that all relevant eye-movements can be precisely and reliably captured is made, another potent issue is which eye-movement to take into account. Firstly, the number of eye-movements in response to a stimulus cannot be predicted; to our knowledge there are no studies that show any statistically significant results in this respect. Therefore, by fixing the number of analysed eye-movements, bias is introduced in the selection process.
It is unknown how Thomason et al. (1980) selected the relevant eye-movement(s). In the study by Elich et al. (1985, p. 622), the authors specify that “eye-movements were recorded from the moment of asking the image-evoking question up through subject’s description of the images experienced in response to the question”; however, the process of selecting the eye-movement judged as relevant to the question is also unknown. Similarly, Poffel and Cross (1985) provide no information on the matter.
In a study related to the PRS, Falzett (1981) selected the eye-movement prior to the acknowledgement of the subject that an internal response has been reached. This process was replicated by Farmer et al. (1985). Gumm et al. (1982) and Sandhu (1991) assessed the first eyemovement following the end of each question, while Wertheim et al. (1986) recorded the first eye-movement after the subject was asked to recall the stimuli as well as the last eye-movement 25 before the subject’s acknowledgement of their internal response. Recording the first eyemovement after the end of the question was earlier done by Cheney et al. (1982); in their study, multiple eye-movements were regarded as a separate event and selection was not attempted.
Ellickson (1983) made a distinction between occurrences of one and two eye-movements; in the latter case, the second eye-movement was selected. There are several flaws these studies have in common.
There is an implicit assumption that eye-movements (or at least the relevant one) occur after the end of the question, which is not necessarily true and this is supported by Cheney et al. (1982) who pointed out that often the subject’s eyes will shift before the end of the question.
Bandler and Grinder (1979) also suggest that some eye-movements may reflect a speech preparation, rehearsal, or translation process or the first eye-movement may reflect the “lead” system, i.e. the representational system that the subject uses to bring the representation into consciousness; in the example offered earlier, the kinaesthetic system (the feeling of sinking into warm bath water) would be the lead system.
In the case of asking the subject to acknowledge reaching an internal response before verbalising, it is possible that the last eye-movement corresponds to a process related to this acknowledgement.
Buckner and Reese (1987) recorded whether any eye-movement that matched the expected modality was present when asking their subjects if they were aware of VAK components in their thought. While the EAC model does not define a specific selection process, it is questionable if this methodology can yield objective results.
Baddeley and Predebon (1991) recorded a series of eye-movements in each part of their study;
the two models they used are shown diagrammatically in Figure 3 together with the other approaches. This is an improvement over previous studies in that it attempts to record multiple eye-movements. However, there are three fundamentally problematic assumptions that cannot
be predictably satisfied. Those are:
1. All subjects will always perform the same amount of eye-movements (no selection criteria are discussed).
2. The representational system targeted by the question will occur on the same eyemovement instance for all subjects. In reality, variations of cognitive and physiological responses can be expected in different people answering the same question.
3. Eye-movements have a one-to-one correspondence to internal representations or processes. In reality, one cannot be certain what these processes are; also pointed out by Cheney et al. (1982).
Recently, Burke et al. (2003) video recorded and scored all eye-movements and performed pattern analysis on sets of two, three, and more than three eye-movements. Cheney et al. (1982) reported that eye-movements will often transpire before the interviewer has reached the end of the question as early as 1982 and it is therefore surprising that no studies up to Burke et al.
(2003) take this into account.
FIGURE 3: DIFFERENT MODELS OF EYE-MOVEMENT SELECTION FOUND IN LITERATURE. THE BLUE
ARROWS INDICATE WHERE EYE-MOVEMENT MEASUREMENTS WERE MADE FOR EACH MODEL. THE
GREY ARROWS DESIGNATE OTHER POSSIBLE INSTANCES OF THE CHOSEN EYE-MOVEMENT. DOTS ARE
USED TO DESIGNATE AN INTERVAL WITHIN WHICH THE EYE-MOVEMENTS CAN OCCUR.
27 As mentioned earlier, the EAC model predicts that eye-movement patterns are observed in all individuals regardless of handedness and a generalisation is offered for normally-organised right-handed people (Bandler and Grinder 1979, p. 25; Figure 1). Given this is an explicit generalisation, it will not hold true for all right-handed people and it is thus not sufficient to screen for right-handed people. In order to investigate these claims it is necessary to test whether idiosyncratic patterns exist within any given individual. This idiosyncratic case was only tested by Burke et al. (2003) who had partially supportive results, while all the other studies used this generalised form in order to interpret the selected eye-movements.
INTERPRETING AND ANALYSING EYE-MOVEMENT DATA
Another interesting aspect of the studies that has never been commented on is the statistical variance of the results reported. Cheney et al. (1982) report no eye-movement 32% of the time while 18.9% of the responses were multiple eye-movements and not analysed. In the same study, eye-movements to the left and up and left were only 14.75% and 10.3% respectively. Poffel and Cross (1985) reported no eye-movement 50% of the time – vastly different results to those of Thomason et al. (1980) despite the similarity in methodologies.
Farmer et al. (1985) reported 49.6% baseline eye-movements with upwards movements coming second at 37% – a significant difference, especially in light of the roughly equal results reported on all VAK components by Thomason et al. (1980). Wertheim et al. (1986) did not include numerical data but report a majority of auditory responses regardless of question type. Dooley and Farmer (1988) and Farmer et al. (1985) report 44% stares for their aphasic subjects and 40% auditory for their control subjects. Finally, Baddeley and Predebon (1991) report 40% and 41.9% leftwards eye-movements in study one and two respectively.
The aforementioned incongruences observed in the results of the studies can lead to two possible logical conclusions: 1) eye-movements are random; or 2) there are variables that have not been considered (or perhaps discovered) and controlled for.