Model-free non-rigid head pose tracking by joint shape and pose estimation

Abstract

Head pose estimation under non-rigid face move- ment is particularly useful in applications relating to eye-gaze

tracking in less constrained scenarios, where the user is

allowed to move naturally during tracking. Existing vision- based head pose estimation methods often require accurate

initialisation and tracking of specific facial landmarks, while

methods that handle non-rigid face deformations typically

necessitate a preliminary training phase prior to head pose

estimation. In this paper, we propose a method to estimate

the head pose in real-time from the trajectories of a set of

feature points spread randomly over the face region, without

requiring a training phase or model-fitting of specific facial

features. Conversely, our method exploits the 3-dimensional

shape of the surface of interest, recovered via shape and

motion factorisation, in combination with Kalman and par- ticle filtering to determine the contribution of each feature

point to the estimation of head pose based on a variance

measure. Quantitative and qualitative results reveal the capa- bility of our method in handling non-rigid face movement

without deterioration of the head pose estimation accuracy.