An Investigation Into Features For Multi-View Lipreading

Adrian Pass, Jianguo Zhang, Darryl Stewart

Research output: Chapter in Book/Report/Conference proceedingConference contribution

17 Citations (Scopus)
271 Downloads (Pure)

Abstract

For the first time in this paper we present results showing the effect of speaker head pose angle on automatic lip-reading performance over a wide range of closely spaced angles. We analyse the effect head pose has upon the features themselves and show that by selecting coefficients with minimum variance w.r.t. pose angle, recognition performance can be improved when train-test pose angles differ. Experiments are conducted using the initial phase of a unique multi view Audio-Visual database designed specifically for research and development of pose-invariant lip-reading systems. We firstly show that it is the higher order horizontal spatial frequency components that become most detrimental as the pose deviates. Secondly we assess the performance of different feature selection masks across a range of pose angles including a new mask based on Minimum Cross-Pose Variance coefficients. We report a relative improvement of 50% in Word Error Rate when using our selection mask over a common energy based selection during profile view lip-reading.
Original languageEnglish
Title of host publication2010 17th IEEE International Conference on Image Processing (ICIP)
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages2417-2420
Number of pages4
ISBN (Electronic)978-1-4244-7993-1
ISBN (Print)978-1-4244-7992-4
DOIs
Publication statusPublished - Sep 2010
Event2010 IEEE 17th International Conference on Image Processing - Hong Kong, Hong Kong
Duration: 26 Sep 201029 Sep 2010

Conference

Conference2010 IEEE 17th International Conference on Image Processing
CountryHong Kong
CityHong Kong
Period26/09/201029/09/2010

Bibliographical note

ISSN: 1522-4880

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Signal Processing
  • Software

Fingerprint Dive into the research topics of 'An Investigation Into Features For Multi-View Lipreading'. Together they form a unique fingerprint.

Cite this