Human emotional state recognition using 3D facial expression features

In recent years there has been a growing interest in improving all aspects of the interaction between human and computers. Emotion recognition is a new technique based on affective computing that is expected to significantly improve the quality of human-computer interaction system and communications. Most existing works address this problem using 2D features, but they are sensitive to head pose, clutter, and variations in lighting conditions. In light of such problems, two 3D visual feature based approaches are presented, the 3D Gabor feature and the 3D elastic body spline features from video sequences. The most significant contributions of this work are detecting and tracking fiducial points automatically from video sequences to construct a generic 3D face model, and the introduction of EBS deformation features for emotion recognition. These methods open a new research direction for human computer communication with applications to security systems, the intelligent home, a learning environment, and educational software.