«BY GEORGIOS DIAMANTOPOULOS A THESIS SUBMITTED TO THE UNIVERSITY OF BIRMINGHAM FOR THE DEGREE OF DOCTOR OF PHILOSOPHY DEPARTMENT OF ELECTRONIC, ...»
To be clear, the term non-visual tasks refers to those tasks which do not explicitly rely on vision to be performed. One example of such an experiment is the “Hollywood Squares” paradigm, where the subject is shown a grid of four squares. Sequentially, an object is placed into one of those four grid squares and a fact is presented auditorily. The grid is then taken away and the subject is asked to answer a question relevant to one of the facts previously presented. Since the objects displayed in the grid can be identical, any visual significance is removed and thus this task does not rely on vision itself at any stage; as such, it can be termed as non-visual. In this task, it was found (Richardson and Spivey, 2000) that eye-movements played a key role in encoding the information presented in the auditory modality. Similarly, the term non-visual eye-movements refers to eye-movements concerned with non-visual tasks. Alternatively, visual eye-movements may be defined as eye-movements whose purpose is to change the visual stimulus falling on the fovea and non-visual as those eye-movements that are a result of neuro-physiological events and are not associated with vision.
Interestingly, models which make use of eye-movements but which are not related to visual tasks (at least not extrospective visual tasks) appeared in the last few decades. The Neuro-Linguistic Programming (NLP) Eye-Accessing Cues (EAC) model was introduced by Bandler and Grinder (1977) and suggests that the direction of non-visual eye-movements indicates the modality (visual, auditory, kinaesthetic) of the subjective experience a person is currently accessing.
9 Simply said, when a person1 is looking down and to their right, they are accessing a feeling associated with the experience they are talking about or examining internally.
While it cannot be denied that eye-movements are hard-wired to brain function, the NLP EAC model was not scientifically validated by its authors. As it will be presented in detail in Chapter 2, several studies appeared at later dates that attempted to (dis-)prove the model but whose results suffered from severe methodological and experimental flaws. The use of direct viewing to record, select and rate the eye-movements has been the genesis of very significant limitations that will be discussed in full detail in Chapter 2.
It quickly becomes apparent that studies relevant to the NLP EAC model and other such models would have benefited by the use of eye-tracking systems. However, selecting a suitable eyetracking system for this task does not prove to be as easy. This is for several reasons, the most important one being that eye-trackers to date were designed to track visual eye-movements.
Whilst the classification of visual versus non-visual eye-movements is not significant in itself, visual eye-movements are normally bound by a much smaller field of view. By contrast, when a person is thinking his or her eyes will usually shift to one of the extremities of the eye socket (regardless of the direction). Thus, if the person was asked to consciously look in the same direction indicated by this shift, he or she would have turned his or her head and performed a much smaller eye-movement to reach the target; this behaviour is largely undocumented. What is, however, documented is the tendency of subjects to shift their eyes when asked to answer a question (not related to a visual task) and before they return to looking at the interviewer.
Therefore, it is no surprise that eye-trackers designed to track visual eye-movements fail to track non-visual eye-movements of such extremities and the need for development of an eye-tracker that is able to track such eye-movements is introduced. The technical detail of this inadequacy as well as a review of recent video-based eye-trackers will be presented in Chapter 3.
In designing and implementing this novel eye-tracker several non-functional requirements must be considered because of the nature of the particular application(s). For example, in the context of the EAC model and other models where the interviewer-subject relationship must be This is a generalization offered by Bandler and Grinder (1979) for a cerebrally normally-organized righthanded person. The EAC model is explained in more detail in Chapter 2.
10 characterized by harmony (often referred to as rapport), precision or accuracy may not be the key requirement for a particular research application. Instead, the invasiveness of the eyetracker is a key requirement, as any discomfort experienced by the subject will “break” rapport.
Some of the aforementioned eye recording or tracking devices, especially the early ones, required that the subject’s eyelids are pulled open with adhesive plaster or clamps and the device makes contact to the eyeball in order to perform the measurements (Carpenter, 1988;
Yarbus, 1967). Other devices, even modern ones, require that the chin is fixated using a chinrest or bite bar. All of these requirements would most certainly increase the subject’s discomfort during the experiment and thus render these eye-trackers unsuitable for such applications.
Several factors can influence the invasiveness of an eye-tracker such as: a) whether it requires contact to the eyeball or other parts of the body, b) whether it restricts any type of movement (e.g. movement of the head) and c) if it is mounted on the head or body, how much it weighs and how long it takes before wearing the eye-tracker becomes uncomfortable for the user.
The thesis presented here is concerned with the development of the Robust Eye-Accessing Cues Tracker (REACT) whose main requirements are to maintain a high level of accuracy while tracking non-visual eye-movements as well as minimize invasiveness and cost, which is always a concern. Finally, while initially targeted to non-visual eye-movement tracking for research applications such as the NLP EAC model, it is desirable that the REACT eye-tracker can be adapted to other eye-tracking applications and be easy to assemble.
The REACT eye-tracker is head-mounted but very lightweight (approx. 60 grams) and is thus minimally invasive. A head-mounted approach was chosen over a remote camera one as to minimize cost and increase the resolution of the captured images and consequently the accuracy viable by the eye-tracker. Moreover, a remote camera requires that the head is tracked and not only does that increase the computational complexity but it also decreases the ability of the eyetracker to track the aforementioned extreme eye-movements because of limitations imposed by the camera viewing angle. Several of the design choices made will be contextualized in Chapter 3, where a comprehensive review of recent eye-trackers is given.
On the hardware-level, the eye-tracker works by illuminating the eye with near-infrared light while blocking most of the visible light spectrum with an infrared filter imposed over the camera 11 lens; the hardware design is discussed in detail in Appendix A. This produces the dark-pupil effect (discussed in more detail in Chapters 3 and 4) which allows for the easy detection of the pupil and, in a sense, offloads some of the processing to the hardware. The captured images are then processed in software and a set of three eye features are detected: the pupil centre and contour, the iris radius and the location of the eye corners. Finally, features are combined in order to calculate the 2D gaze angle.
In terms of the thesis organization, Chapter 2 is a brief introduction to Neuro-Linguistic Programming and the Eye-Accessing Cues model specifically and a complete and detailed critical review of past EAC model research. At least six out of ten studies that have investigated the EAC model since its first introduction in 1977 (Grinder, DeLozier and Bandler, 1977) have reported unsupportive results. In this review of past research, these studies and their respective experimental methodologies are examined and the reliability of their results is assessed. The review is extended by presenting findings from other relevant eye-movement research while discussing their relevancy to and implications for the EAC model. Thus, in this chapter, it is argued that there is substantial ground for further research into the EAC model and identify the requirements that should inform this work, a significant one of which is using an eye-tracker to perform any further research. In doing so, the motivation of this research work will be established in further detail.
After having established this motivation, a review of existing eye-tracking systems is presented in Chapter 3, with a view towards discussing their limitations in tracking extreme non-visual eyemovements, the requirements of a non-visual eye-tracker as well as fundamental decisions that have informed the design of the REACT eye-tracker. Overall, Chapter 3 aims to give a brief overview of other systems regardless of their modi operandi but go into detail where relevant to this research work.
Chapter 4 then explicates the algorithms involved in detecting the eye features and combining them to calculate the 2D gaze using a detailed description of the mathematical and computer vision concepts as well as several illustrations where appropriate. In Chapter 5, the performance of the eye-tracking hardware and the eye feature detection algorithms presented in Chapter 4 is evaluated and the results are discussed.
12 Chapter 6 presents a pilot study specifically designed to apply the eye-tracker in a real-world application which will serve as a case study for this work. The full transcript with the results visualised is included as Appendix B and selected parts are extracted to facilitate discussion in Chapter 6.
Finally, Chapter 7 offers some concluding remarks and discussion of future work. Appendix C includes a list of already published journal and conference papers and a paper for submission to the Special Issue of the Signal, Image and Video Processing Journal entitled “Unconstrained Biometrics: Advances and Trends”.
CHAPTER 2: AN INTRODUCTION TO THE NLP EAC MODEL AND
CRITICAL REVIEW OF PAST RESEARCHThis chapter is a brief introduction to Neuro-Linguistic Programming (NLP) and the EyeAccessing Cues (EAC) model specifically. Parts of this chapter have been published in the proceedings of the First International NLP Research Conference held at the University of Surrey, UK, on 5th July 2008 (Diamantopoulos et al., 2008). For a more in-depth introduction to NLP and its models, the interested reader is referred to the relevant literature (Bandler and Grinder, 1975; Grinder and Bandler, 1976; Bandler and Grinder, 1979; Dilts and DeLozier, 2000).
Since its introduction in 1977, the EAC model has been investigated in ten studies. While six of these studies report unsupportive results, a clear conclusion as to the validity of the model has not been reached. Each one of these studies is considered in the first part of this chapter and it is shown that, upon careful examination, the respective experimental methodologies were based on assumptions informed by an incomplete or erroneous understanding of the EAC model that could have significantly influenced the experimental results. The reliability of the results can be further impacted by the absence of modern eye-tracking equipment to support the inherently complex task of reliably recording, selecting and rating eye-positions. Further doubt is raised as to the validity of the results as most studies reported statistically significant results (whether in favour of the model or not) and yet, the correlations reported are not in agreement across studies.
Review efforts have been made before (Sharpley, 1987; Heap, 1988; Richardson and Spivey 2004), where NLP is criticised as unsupported by research efforts. However, these reviews are drawn from the reported results of the referenced studies rather than a critical review of the literature with strong background knowledge of the models in question. Further, Heap (1988) bases his conclusions largely on results reported by masters’ dissertational theses; of his large list of 66 references, 36 are dissertations. Thus, the present review is restricted to peer-reviewed publications that concern the EAC model only.
Further extending our survey in the second part of this chapter, recent eye-movement research from other fields is presented and its relevancy to and implications for the EAC model are discussed. Thus, it will become apparent that there is no published research that directly proves or disproves the EAC model and there is substantial ground for further research. Finally, in the 14 last part, drawing from the strengths and weaknesses of past research in the EAC model and the research findings from eye-movement and cognition research, this chapter attempts to identify the requirements that should inform future research.
NEURO-LINGUISTIC PROGRAMMING AND REPRESENTATIONAL SYSTEMSThe roots of Neuro-Linguistic Programming (NLP) descend from the work of Richard Bandler and John Grinder in the early 1970s. Their first seminal work (Bandler and Grinder, 1975) was based on their study of Virginia Satir and Fritz Perls, a family therapist and the father of Gestalt therapy respectively, and introduced the meta-model. The meta-model is a linguistic model “about the way language functions in modelling the world” (Tosey, 2006).
Bandler and Grinder continued their work by studying Milton Erickson, a successful hypnotherapist, and further publishing two books about his hypnotic techniques (Bandler and Grinder, 1975; Grinder, DeLozier and Bandler, 1977), as the Milton model.
Integral to NLP is the notion of representational systems; as defined by Bandler and Grinder (1979, p. 14) the representational system is the sensory system that a representation of a
person’s subjective experience is held or accessed in: