WATSON: Real-time Head Tracking and Gesture Recognition
Principal Investigators:
Watson can track rigid objects in real-time with 6 degrees of freedom using a tracking framework called Adaptive View-Based Appearance Model.  The tracking library can estimate the pose of the object for a long period of time with bounded drift.  Our main application is head pose estimation and gesture recognition using a USB camera or a stereo camera.
Our Approach:
Our approach combines an Adaptive View-based Appearance Model (AVAM) with a robust 3D view registration algorithm. AVAM is a compact and flexible representation of the object that can be used during the tracking to reduce the drift in the pose estimates. The model is acquired online during the tracking and can be adjusted according to the new pose estimates.  Relative poses between frames are computed using a hybrid registration technique which combine the robustness of ICP (Iterative Closest Point) for large movement and the precision of the normal flow constraint.  The complete system runs at 25Hz on a Pentium 4 3.2GHz.
Watson tracking library is available for download (for research purpose only). This library also include a module for head gesture recognition.  To download it, please send an email to Louis-Philippe Morency (lmorency@csail.mit.edu).  You can look at the documentation of Watson:  Watson23.pdf .
Related Publications:
  1. Louis-Philippe Morency, Candace Sidner, Christopher Lee, Trevor Darrell, Contextual Recognition of Head Gestures, Proceedings of the International Conference on Multimodal Interactions, 2005.
  2. Louis-Philippe Morency, Ali Rahimi, Trevor Darrell,Adaptive View-based Appearance Model, Proceedings IEEE Conf. on Computer Vision and Pattern Recognition, 2003.
  3. Louis-Philippe Morency, Trevor Darrell, Stereo Tracking using ICP and Normal Flow, Proceedings Int. Conf. on Pattern Recognition, 2002.
  4. Louis-Philippe Morency, Ali Rahimi, Neal Checka, Trevor Darrell, Fast Stereo-Based Head Tracking for Interactive Environment, Proceedings of the Int. Conference on Automatic Face and Gesture Recognition, 2002.
  5. Ali Rahimi, Louis-Philippe Morency, Trevor Darrell, Reducing Drift in Parametric Motion Tracking, Proceedings of the International Conference on Computer Vision, 2001.