A hierarchical framework for high resolution facial expression tracking

Xiaolei Huang, Song Zhang, Yang Wang, Dimitris Metaxas, Dimitris Samaras

Research output: Contribution to journalConference articlepeer-review

33 Scopus citations


We present a novel hierarchical framework for high resolution, nonrigid facial expression tracking. The high quality dense point clouds of facial geometry moving at video speeds are acquired using a phase-shifting based structured light ranging technique. To use such data for temporal study of the subtle dynamics in expressions and for face recognition, an efficient nonrigid facial tracking algorithm is needed to establish intra-frame correspondences. In this paper, we propose such an algorithmic framework that uses a multi-resolution 3D deformable face model, and a hierarchical tracking scheme. This framework can not only track global facial motion that is caused by muscle action, but fit to subtler expression details that are generated by highly local skin deformations. Tracking of global deformations is performed efficiently on the coarse level of our face model with one thousand nodes, to recover the changes in a few intuitive parameters that control the motion of several deformable regions. In order to capture the complementary highly local deformations, we use a variational algorithm for non-rigid shape registration based on the integration of an implicit shape representation and the Free Form Deformations (FFD). Due to the strong implicit and explicit smoothness constraints imposed by the algorithm, the resulting registration/deformation field is smooth, continuous and gives dense one-to-one intra-frame correspondences. User-input sparse facial feature correspondences can also be incorporated as hard constraints in the optimization process, in order to guarantee high accuracy of the established correspondences. Extensive tracking experiments using the dynamic facial scan of five different subjects demonstrate the accuracy and efficiency of our proposed framework.

Original languageEnglish (US)
Article number1384811
JournalIEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
Issue numberJanuary
StatePublished - 2004
Event2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2004 - Washington, United States
Duration: Jun 27 2004Jul 2 2004

All Science Journal Classification (ASJC) codes

  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering


Dive into the research topics of 'A hierarchical framework for high resolution facial expression tracking'. Together they form a unique fingerprint.

Cite this