«BY GEORGIOS DIAMANTOPOULOS A THESIS SUBMITTED TO THE UNIVERSITY OF BIRMINGHAM FOR THE DEGREE OF DOCTOR OF PHILOSOPHY DEPARTMENT OF ELECTRONIC, ...»
In this project, it was thought prudent to include a basic evaluation of the eye-tracker’s usability that could highlight any major problems with its usability and the comfort of the subjects that would render it unusable for the applications in question. Thus, the basic questionnaire shown below () was distributed to the subjects that have taken part in all experiments; the results are shown in Table 29. Question one aimed to establish the overall level of comfort that subjects perceived themselves as experiencing during the experiments. Questions two and three were targeted towards establishing how aware the subjects were of the eye-tracker’s presence on their head and its presence in their visual field respectively. Finally, question four asked the subjects whether they would hypothetically participate in a longer experiment (eight out of nine subjects made only occasional use of the eye-tracker with each session lasting less than five minutes) and in case of a negative answer, question five enquired as to whether this was relevant to a usability aspect of the eye-tracker (such as it being too heavy etc.).
As can be seen from Table 29, the majority of subjects (seven out of nine) answered they were generally comfortable wearing the eye-tracker. Subject five gave a medium rating (neither comfortable not uncomfortable) and only subject five and six gave a low rating on this question (2 – uncomfortable). The average across subjects is 3.6 points.
In this small sample, a rating of 4 (comfortable) on question one corresponded to a rating of 1-2 in question two (very unaware/unaware of the eye-tracked being placed on their head). Subjects one and nine who gave a rating of 4 on question one answered that they were neither aware nor unaware of the eye-tracker being placed on their head. In terms of invasiveness, a neutral rating can still be considered as favourable as it means that no discomfort that could distract the subject from the task at hand was caused. Of course, it also means that subjects that gave a neutral rating
Q5. If you answered no in the above question, would that be because of any aspect of the eyetracker's usability (e.g. feels too heavy on the head, is tiring to wear etc.)?
Question three received much more favourable answers than question two; four out of nine subjects gave a rating of 1 (very unaware), four out of nine subjects gave a rating of 1 (unaware) and subject two only gave a rating of 2 (neither aware nor unaware).
The results are certainly interesting; in question two, the rating was never higher than 4 (aware) but in question three the ratings did not exceed 3 (neither aware nor unaware). It is hypothesised that this is a favourable result; to be clear, it is hypothesised that unfavourable
ratings in question three would have a more significant effect to the invasiveness of the eyetracker than unfavourable ratings in question two. This hypothesis is made because:
During a longer experiment where subjects would have time to become comfortable with wearing a foreign device on their head, they would slowly drift away from the consciousness of the eye-tracker and immerse themselves in the task or conversation central to the experiment.
While movements of the head would not change the feeling of wearing the eye-tracker, if the eye-tracker significantly blocked their field of view, the subject would be reminded of its existence as their eyes moved.
162 The above reasoning, is reflected in the results as the subjects who gave an unfavourable rating on question one, gave an unfavourable rating in question two and a favourable rating in question three. It would thus appear that they judged the overall invasiveness of the eye-tracker based on the feeling of wearing it versus the eye-tracker camera being somewhat in their visual field.
In fact, subject five who gave a neutral rating on question one, at the end of the questionnaire, commented that he gave such a rating and answered “No” to a hypothetical future experiment because he found that his eyes hurt after each experimental session. This strain on the eyes must have been caused by trying to looking at points on the screen that spanned a larger area than comfortable to the subject. Thus, it can only be concluded that this was due to a miscommunication on the experimenter’s part as the ability to reduce the radius of the circle that the points laid on was offered to each subject such that their eyes are not strained at any time.
Similarly, subject six commented that “it felt like the glasses wanted to slip to the end of the nose” and added that it may have been because he had never had to wear glasses before. Other than becoming familiar with the feeling of wearing glasses, from this statement, it can be concluded that it would be helpful to try several different frames for the eye-tracker, including metallic ones and find which frame or frames gathered the best responses from test subjects.
The overall performance of the eye-tracker in terms of invasiveness may be summarised by the comments of subject nine who stated that “overall, the eye-tracker was pretty unobtrusive although I was always aware that it was there”. Even though all subjects reported that they were aware of the eye-tracker’s existence, they found it was comfortable to wear it during the short experimental sessions and that they were rarely aware of the eye-tracker in their visual field.
Thus, in longer sessions when the subject is engaged in the natural task of conversing with the interviewer, the invasiveness of the eye-tracker would be reduced from minimal to negligible.
163CHAPTER 6: CASE STUDY
As explained in the introductory chapter and the literature review, our motivation in building the REACT eye-tracker has been to enable researchers interested in eye-movements that are not generated by outwards visual attention to e.g. tracking a visual target but rather by inwards attention (i.e. thinking) to perform experiments using eye-tracking technology.
Thus, in addition to the extensive evaluation of the eye-tracker in the previous chapter, it was decided to demonstrate the eye-tracker’s applicability to a real-world scenario through a case study, at the same time illustrating the value of using an eye-tracker. For reasons that have been mentioned before and will be mentioned again throughout this chapter, the pilot experiment of this chapter does not attempt to support or disprove the NLP EAC model. The results from the experiment cannot support or disprove the model.
Once again bearing in mind our motivation through the NLP literature, a pilot study was designed where a subject was interviewed by the experimenter on various topics such as education and hobbies, meanwhile recording the subject’s eye-movements, for a period of approximately twenty (20) minutes. The conversation was then manually transcribed and the position in time of each word uttered by both the subject and experimenter was also manually recorded. This process resulted in a rich data set that allowed the precise overlay of eyemovements, in time with speech. The video and audio were recorded simultaneously into the same file and thus synchronisation was taken care of.
In this chapter, the extension of the eye-tracker to continuously track the subject’s eyemovements over time is described in detail. The pilot study experiment and its relation to the NLP eye-accessing cues model are briefly discussed with the aid of selected extracts. Finally, the full data set of the transcribed text with timestamps and overlaid eye-movement classifications are presented in Appendix B.
EYE-TRACKING OVER TIME
In Chapter 3, the algorithms involved in detecting the set of eye features necessary to calculate the 2D gaze of the subject were deconstructed in full detail. Regardless of the inherent complexity in calculating these low-level features, they provide little information to a simple user
While the 2D gaze calculated using these features can be considered a higher-level output of the eye-tracker, it still serves poorly as the main output of the eye-tracker for several reasons discussed below.
First of all, it generates a large amount of data samples, far larger than what can be manually analysed by the average user. At 29.97 captured frames per second, 59.94 fields are analysed by the eye-tracker per second. That is almost 3596 data samples per minute.
Not only is the amount of data samples too large to manually process, but it may also be considered as unnecessary. The speed at which eye-movements are detected is limited by the capturing rate, which is dictated by the camera. For example, smooth pursuit11 studies that require very high sampling rates could not be conducted because the camera is only capturing
29.97 frames/59.94 fields per second and would require much higher sampling rates (in the order of 250-500Hz) that only high-end commercial eye-trackers, such as the SR-Research EyeLink-II, can offer due to the inherent hardware costs. Thus, in the range of eye-movements that can be tracked by the REACT eye-tracker, the experimenter will most likely be interested in fixations, during which the eye is fixated for a certain amount of time at one location12 and what will be referred to as “search paths” in this chapter.
Before even designing the REACT eye-tracker and while conceiving its requirements, a camcorder was used to record the eye-movements of a subject during short interviews. After transcribing the conversation and using the video time stamps to automatically overlay the eyemovements on the text, it was noticed that the subject would often perform several sequential eye-movements with small pauses in between before fixating at a final position and answering the question. For example, when asked about the nature of a particular experience, the subject may have looked right, paused for a very brief amount of time, then looked up and to the right Smooth pursuit is defined as the eye-movements that occur when the eyes closely follow a moving 11 object.
12 Normally, fixations refer to the point in the world that the subject is fixating upon. However, in this discussion, fixations simply refer to when the eye has moved off the centre and remains fixed for some time.
Search paths although explicitly defined here are not a new observation. In NLP, they have been modelled through the notions of lead and representational system as introduced in the literature review of Chapter 2. However, in research related to the NLP EAC model, only a limited set of researchers have taken into account “search paths” or sequences of eye-movements such as Baddeley and Predebon (1991). For further information, see Chapter 2.
Thus, fixations and search paths are the eye-movements that the REACT eye-tracker is targeted
at selecting. This selection is done as follows (illustrated in the state diagram of Figure 38):
1) The current pupil position is detected and the corresponding 2D gaze angle is calculated.
2) If the subject is looking straight ahead, the frame is skipped. Otherwise, it is classified into one of the eight categories listed below, the class is recorded and a count is maintained.
3) On the next frame, if the classified position is the same as the one recorded, the count is updated. If the subject is now looking straight ahead, the count is reset.
4) If the count reaches a threshold, the frame is selected. This allows fixations to be selected.
5) When a frame is selected, is set to a lower threshold and the algorithm continues onto the next frame. This allows for search paths to be detected and its component frames selected.
Eye-movements are classified into eight (8) classes as per the classes introduced in NLP (Bandler and Grinder, 1979; Figure 1); specifically up, up and to the left, up and to the right, left, down, down and to the left, down and to the right, right. For a 2D gaze angle and, a class is
assigned as follows (illustrated visually in Figure 39):
As before, all processing takes place off-line.
A tolerance angle of is used for the and classes on the bottom hemisphere (180° to 360°) to accommodate for the reduced vertical resolution that is a direct result of separating each frame into two fields (see de-interlacing in Chapter 3). The tolerance angle is further reduced to for the top hemisphere of classes and (0° to 180°) to accommodate for the skew that is a direct result of the camera point upwards.
As analysed in Chapter 4, a calibration frame is required to initialize the eye-tracker to the subject. This is done by asking the subject to look straight ahead with his or her chin approximately parallel to the floor (posture is adjusted with the help of the experimenter). The eye-tracker automatically re-initializes itself to accommodate for changes in the location of the corners on every frame where the pupil position is within a contour twice as large as the initial pupil contour. In most sequences the latter contour is a little smaller than the iris. Reinitialization is not performed if the eye-tracker has re-initialized in the last 60 frames (approximately 10 seconds).
During re-initialization, the “initial pupil position” used to calculate the 2D gaze angle is
updated to from the new corner locations and such that:
This calculation is based on the assumption that the geometry acquired during the calibration of the eye-tracker as described by the distances between the eye corners and the initial pupil position is correct.
CASE STUDY PILOT EXPERIMENTThe pilot experiment designed for this case study aimed to be a simplistic approximation of an experiment that would be performed to further investigate the correlation of eye-movements and thinking modalities or in other words, the NLP eye-accessing cues model.
The basic premise of such an experiment would be to investigate whether the eye-movement class is correlated to the question type or verbal predicates used by the subject when answering.