«BY GEORGIOS DIAMANTOPOULOS A THESIS SUBMITTED TO THE UNIVERSITY OF BIRMINGHAM FOR THE DEGREE OF DOCTOR OF PHILOSOPHY DEPARTMENT OF ELECTRONIC, ...»
Creating an elliptical model of the typical extreme eye-movements (U, UR, R, DR, D, DL, L, UL) Finding the closest point to the current pupil position, on ellipse.
Calculate the distance between and the centre of ellipse and compare it against the corresponding distance between the centre and the typical eye-movement point for each of the two candidate classes.
Return the class that is closest to.
In conclusion, in this chapter, a simplistic case study was successfully designed and performed in order to demonstrate the usefulness of using the eye-tracker in an experiment relating to nonvisual eye-movements. Finally, the eye-tracker demonstrated satisfactory performance in selecting and rating the eye-movements throughout the case study.
Neuro-Linguistic Programming and the Eye-Accessing Cues model have helped form the motivation behind this thesis; as discussed in Chapter 2, the EAC model has been the centre of research attention for almost as long as video-based eye-trackers have existed, with each study displaying one or more significant methodological errors, two of which have been pivotal.
The first is the lack of development of a questioning methodology with strong academic foundations, with the emphasis on “development”. From the review of Chapter 2, one can be certain that eye-movements are directly linked to brain activity but we have yet to scientifically establish whether eye-movements in conversation can be consistently triggered and much more importantly, whether a questioning methodology that goes to depth, rather than breadth as all research studies have done to date, will consistently elicit the same or similar-enough eyemovements. These are very important research questions that need to be answered before researchers attempt to directly prove or disprove the EAC model itself and they are not trivial to answer. These limitations were also established through the presentation of this review at the First International NLP Research Conference and later through its publication in the conference proceedings (Diamantopoulos et al., 2008).
Thus, in light of the inconsistent results and therefore tentative conclusions of past research as well as recent aforementioned developments in academia, it was established that the EAC model is in fact a very complex model which requires further research attention and rigorous examination if it is going to be proved or disproved.
Research in the EAC model has stopped for almost two decades and now is a good time to continue these efforts, as eye-tracking technology has matured and become more affordable too, which brings us to the second pivotal methodological error: the lack of use of a reliable and precise method to record, select and rate eye-movements.
As discussed in Chapter 3, to date, no eye-tracker had fulfilled all the requirements of tracking non-visual eye-movements15.
First of all, existing eye-trackers have, without exception, been designed to track visual eyemovements (within 30° of angular range) and are severely limited when faced with the task of tracking non-visual eye-movements which tend to occur outside the regular field-of-view (30°°). A formal explanation of this empirical conclusion cannot be found in the literature though an intuitive conclusion is easily deduced; in order to minimize strain of the eyes and maximize visual acuity, people tend to turn their heads in the general direction of the observed object and accurately shift their eyes to bring it in focus.
Secondly, invasiveness is a major concern in this particular application as it may “break” the rapport between the subject and experimenter. Such a link seems particularly plausible in light of recent research; in their experiments, Tognoli et al. (2007) found that the phi complex is a brain rhythm that serves social functions of the brain (both independent and those requiring coordination between people) and produces wave patterns similar to those of mirror neurons.
Thus, the invasiveness of the eye-tracker can significantly affect the results of the study.
Traditionally, the invasiveness of an eye-tracker is related to physiological variables such as whether it is mounted on the subject’s body or whether the subject’s movement is restrained through a device like a chin rest or a bite bar. In this thesis, it was updated to include a major psychological variable: whether the subject is aware of being watched or not. This awareness was suggested to be mainly affected by three factors: a) the feeling of a foreign device on the subject’s head, b) its existence in the subject’s visual field and c) by the calibration procedure which inevitably highlights the non-natural conditions of an eye-tracking experiment. Taking all these variables into consideration, a remote eye-tracker which is the least invasive in the old definition may not be the least invasive in the updated form of invasiveness.
Additionally, requirements of practical significance were determined to be important for this research. Specifically, maintaining low-assembly cost, ease of use and transparency in calibrating the eye-tracker to the subject is vital to making eye-tracking technology more accessible and ensuring its adoption in such research. Also an important requirement of practical significance is Eye-movements not concerned with viewing or tracking an object in the external environment.
15 185 the viability of tracking the eyes of both the subject and interviewer. As the technology becomes commonplace and experimenters become more ambitious in terms of how technology can facilitate their work, it is essential to minimise the required effort to setup such experiments.
Finally, in light of the aforementioned development in requirements by researchers, the ability to extend the eye-tracker to 3D may prove useful.
Thus, in Chapter 2, the need for a specific type of eye-tracker was established and in Chapter 3, current eye-trackers were surveyed in search of one that would fulfil all of the above requirements. Failing to find one, the main objective of the current research work was established and in Chapter 4, a novel set of feature extraction algorithms were presented for extracting the location of the pupil, the iris radius and location of eye corners from images taken from an actively-illuminated head-mounted eye-tracker. This eye-tracker was based on a lowcost hardware design previously published by Babcock and Pelz (2004) thus satisfying the costrelated requirement early on.
First, the pupil is converted to binary using simple global thresholding. Connected components in the image are found using the common labelling algorithm and an ellipse is fitted on each component to find the one with the best fit and thus select the pupil. The contour is then refined using an active contour (also known as snake), before the final pupil location is calculated. Then, the iris radius is calculated using an adaptation of edge strength by Zhou and Pycock (1997), originally developed to segment images of cells. The eye corners are located in the image by making use of the partial x- and y- derivatives and initiating a search in groupings of local maxima. The accuracy of both the iris radius and eye corners search is increased by collecting several candidates and using a novel statistical algorithm to filter outliers that is simple and yet effective. Finally, the 2D gaze direction is calculated using only one calibration point which can be extracted without explicit instructions to the subject and thus the required ease of use and subject calibration transparency mentioned earlier is achieved.
The REACT eye-tracker is one of very few eye-trackers to detect and use the eye corners as reference points to calculate the gaze direction in 2D and the only one to do so from images taken with an actively-illuminated head-mounted eye-tracker; as extensively discussed throughout Chapter 4, the appearance of the eye and the inner eye corner in particular is significantly different close-up than it is from far and the eye corners cease to appear as symmetric. This is a 186 significant development in eye-tracking as the eye corners are always visible, in contrast with the glint which can fall onto the sclera and be very hard or impossible to detect.
Moreover, accurately calculating the iris radius means that by incorporating a 3D model of the eye and providing more calibration points, should future research require it, the eye-tracker can be extended to calculate 3D gaze in a similar fashion to Wang et al. (2005). The use of features versus an appearance-based model also means that the eye-tracker can be adapted to other potential mainstream or sub- applications without much effort.
In Chapter 5, the evaluation of the eye-tracker took place. Specifically, the accuracy of the feature extraction was assessed both independently and as a whole. The eye-tracker achieved a practical level of performance that renders it acceptable for use in the target research application(s). In a comprehensive test of range and accuracy, the REACT eye-tracker was found able to track eyemovements in the complete viewing range of the participants (approx. ±56°/±52°) with an average error of approx. 5° over the whole range and an average error of approx. 1.5° within a narrow field of view (±30°/±20°).
The practicality of the eye-tracker for the target application was further demonstrated in Chapter 6, where a pilot study was designed and served as a case study of a real-world application. Algorithms to select and classify the subject’s eye-movements that are relevant to the experimenter were introduced; the classification was based on the eight (8) classes of the EAC model. Last but not least, a basic evaluation of the eye-tracker’s usability and impact on the perceived comfort of the subjects was performed and yielded satisfactory results.
While a formal study of the improvement over a human rater was not performed, it should follow that the eye-tracker is able to more accurately track eye-movements, especially when performed quickly and in complex sequences. Thus, the REACT eye-tracker is able to facilitate extensive and in-depth research of non-visual eye-movements.
The implications for the computer vision components are several. First, it was demonstrated that reference-based eye-tracking is possible over a large field of view by using the eye corners are reference instead of the glint. Second, the edge strength algorithm used originally for cell segmentation was applied in a completely new context and problem. As these algorithms are further improved, or with better hardware, higher levels of accuracy can potentially be achieved.
Future work can be divided into work related to enhancing the eye-tracker and work related to the investigation of the EAC model.
In terms of further enhancing the eye-tracker, the accuracy of the eye corner detection algorithm can be further improved by developing an algorithm to refine the eye corner locations using the full scale image, by performing a local search within a small window centred at the eye-corner location found at the reduced scale images.
Also, accuracy would be further increased by placing the camera straight in front of the eye, with the lens coplanar to the front surface of the eyes (i.e. if we assume a reference system at the eyeball centre, the camera would be placed at a location – ], with the x-axis going into the person). However, since this would significantly block the subject’s field of view and thus make the eye-tracker significantly invasive, an alternative solution would be to estimate a threedimensional transform that would transform the feature-points to the location they would appear to be at if the camera was placed that way. The latter transform could perhaps be estimated by creating a model of extreme eye-movements and collecting several subject calibration points. As the extra calibration would come at the cost of added invasiveness, one would need to be careful in designing such an algorithm. In addition, such an algorithm would most likely help resolve ambiguities in the classification of eye-movements.
The usability assessment of the eye-tracker (Chapter 5) is a rare occurrence in eye-tracking literature; usually, usability is taken for granted. That is not to say that care is not taken by researchers while designing the eye-trackers to minimise interference. However, no formal study has taken place to assess just how invasive eye-trackers are and how they may affect the person’s behaviour. Thus, eye-tracking system designs have to be no longer limited to functional requirements and pay more attention to non-functional requirements. In the case of the REACT eye-tracker, different frames (including metallic ones) and designs must be tested with a concurrent assessment of their impact in perceived and actual subject invasiveness (perceived being that reported by the subject on questionnaires and actual being any change in behaviour found through scientific means).
188 One possible direction in actually reducing the invasiveness of the eye-tracker is to explore the uses of a wireless technology to transmit the video to a recording device. With the development of low-powered wireless protocols with relatively high bandwidth such as Bluetooth 3.0, this is certainly a possibility that appears feasible and must be explored.
It was also noticed that while the eye-tracker is not dependant on the exact positioning of the camera in relation to the subject’s eye, results tend to be more accurate when the camera is better placed. This is also the case with most, if not all, commercial eye-trackers and two have been tested by the author. It would be useful to develop feedback mechanisms that help the experimenter to adjust the eye-tracker such that the best possible results are achieved.
Of course, the most obvious future work is a full-scale NLP experiment. As discussed earlier, before attempting to directly examine the EAC model again, a questioning methodology needs to be designed, most probably based on procedures from the field of psycho-phenomenology, which is concerned with the exploration of subjective experience. Further, the eye-tracker enables past research with regards to the behaviour of eyes during conversational tasks to be made concrete and accurate conclusions to be drawn.