Perhaps we are scarcely aware of the importance we attribute to a person’s gaze during conversations. During social interactions, we implicitly track each other’s gaze, which conveys unspoken information about our interlocutors’ attention and thought processes. We implicitly track a person’s gaze by combining the head and eye poses into a single gaze direction.
It is no surprise that there is growing academic and commercial interest in the development and use of eye-gaze tracking equipment.
Eye-gaze tracking has been used extensively in human studies, such as for the analysis of eye movement during reading or in the study of attention mechanisms, and in market research. One of the most common uses of eye-gaze tracking is for human-machine interaction, providing an alternative communication and control channel to people with motor impairments.
While eye-gaze control has the potential to provide an alternative remote control channel for everyone, this requires the technology to move out into the ‘wild’, that is, to be robust enough to be applied to everyday life circumstances. In such cases, a user would freely change the head pose, which would need to be tracked in real-time to adjust the gaze direction.
The Computer Vision Group of the Department of Systems and Control Engineering in the Faculty of Engineering, is developing solutions for eye-gaze tracking in the wild through Project WildEye, funded by the Malta Council for Science and Technology (MCST) under the FUSION R&I Technology Development Programme 2016, in collaboration with Seasus Ltd.
The WildEye project aims to develop a low-cost eye-gaze tracker utilising an off-the-shelf webcam to track the user’s gaze as a combination of the user’s eye and head pose under daily-life conditions. This would permit gaze tracking of a user sitting at their desk with normal lighting conditions and moving their head freely.
Naturally, part of the WildEye work is concerned with head pose estimation. Head pose estimation methods often require tracking landmarks on the face. However, in view of the non-rigidity of the face, especially when a user speaks, laughs or expresses an emotion, these landmarks non-rigidly move around with the face movement. Methods that do not compensate for this non-rigid face movement, constrain the user to a rigid facial expression, since any facial movement would otherwise affect the accuracy of the head pose estimation.
The WildEye team has developed a novel algorithm to estimate the head pose robustly despite face variations, such as those due to face expressions, talking or laughing. The developed algorithm estimates the head pose in real-time, without requiring that the images are entirely recorded before the head pose can be estimated, which is known as batch processing. The results of this novel algorithm are shown in the video included with this article which also shows the result obtained by the method of Bregler et al. and by that of Morita and Kanade.
The method of Bregler et al. compensates for non-rigid face movement, but works in batch-mode and hence can only estimate the head pose after all the image frames have been acquired. On the other hand, the method of Morita and Kanade can estimate the head pose in real-time, but does not compensate for the non-rigid facial expressions. Our novel method combines the advantages of the two, and may be seen to produce appropriately stable head pose estimates under large non-rigid face deformations while operating in real-time.
This work forms part of the project R&I-2016-010-T WildEye financed by the Malta Council for Science and Technology through FUSION: The R&I Technology Development Programme 2016. The team working on WildEye is composed of Professor Kenneth P. Camilleri, the Project Coordinator, and Dr Stefania Cristina and Mr Daniel Bonanno from the University of Malta, and Mr Kenneth Bone, Mr David Vella and Mr David Bonello from Seasus Ltd. Interested individuals are invited to visit the project’s website, for more information or to leave feedback.