An Investigation Into Features For Multi-View Lipreading

Adrian Pass, Jianguo Zhang, Darryl Stewart

Research output: Chapter in Book/Report/Conference proceedingConference contribution

18 Citations (Scopus)
363 Downloads (Pure)


For the first time in this paper we present results showing the effect of speaker head pose angle on automatic lip-reading performance over a wide range of closely spaced angles. We analyse the effect head pose has upon the features themselves and show that by selecting coefficients with minimum variance w.r.t. pose angle, recognition performance can be improved when train-test pose angles differ. Experiments are conducted using the initial phase of a unique multi view Audio-Visual database designed specifically for research and development of pose-invariant lip-reading systems. We firstly show that it is the higher order horizontal spatial frequency components that become most detrimental as the pose deviates. Secondly we assess the performance of different feature selection masks across a range of pose angles including a new mask based on Minimum Cross-Pose Variance coefficients. We report a relative improvement of 50% in Word Error Rate when using our selection mask over a common energy based selection during profile view lip-reading.
Original languageEnglish
Title of host publication2010 17th IEEE International Conference on Image Processing (ICIP)
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages4
ISBN (Electronic)978-1-4244-7993-1
ISBN (Print)978-1-4244-7992-4
Publication statusPublished - Sep 2010
Event2010 IEEE 17th International Conference on Image Processing - Hong Kong, Hong Kong
Duration: 26 Sep 201029 Sep 2010


Conference2010 IEEE 17th International Conference on Image Processing
Country/TerritoryHong Kong
CityHong Kong

Bibliographical note

ISSN: 1522-4880

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Signal Processing
  • Software


Dive into the research topics of 'An Investigation Into Features For Multi-View Lipreading'. Together they form a unique fingerprint.

Cite this