Projects per year
Abstract
This paper presents the maximum weighted stream posterior (MWSP) model as a robust and efficient stream integration method for audio-visual speech recognition in environments, where the audio or video streams may be subjected to unknown and time-varying corruption. A significant advantage of MWSP is that it does not require any specific measurements of the signal in either stream to calculate appropriate stream weights during recognition, and as such it is modality-independent. This also means that MWSP complements and can be used alongside many of the other approaches that have been proposed in the literature for this problem. For evaluation we used the large XM2VTS database for speaker-independent audio-visual speech recognition. The extensive tests include both clean and corrupted utterances with corruption added in either/both the video and audio streams using a variety of types (e.g., MPEG-4 video compression) and levels of noise. The experiments show that this approach gives excellent performance in comparison to another well-known dynamic stream weighting approach and also compared to any fixed-weighted integration approach in both clean conditions or when noise is added to either stream. Furthermore, our experiments show that the MWSP approach dynamically selects suitable integration weights on a frame-by-frame basis according to the level of noise in the streams and also according to the naturally fluctuating relative reliability of the modalities even in clean conditions. The MWSP approach is shown to maintain robust recognition performance in all tested conditions, while requiring no prior knowledge about the type or level of noise.
Original language | English |
---|---|
Article number | 6495474 |
Pages (from-to) | 175-184 |
Number of pages | 10 |
Journal | IEEE Transactions on Cybernetics |
Volume | 44 |
Issue number | 2 |
Early online date | 08 Apr 2013 |
DOIs | |
Publication status | Published - 13 Jan 2014 |
Bibliographical note
INSPEC Accession Number:14031829
Date of publication April 8, 2013; date of current version
January 13, 2014
Fingerprint
Dive into the research topics of 'Robust Audio-Visual Speech Recognition under Noisy Audio-Video Conditions'. Together they form a unique fingerprint.Projects
- 1 Finished
-
R1118ECI: Centre for Secure Information Technologies (CSIT)
McCanny, J. V., Cowan, C., Crookes, D., Fusco, V., Linton, D., Liu, W., Miller, P., O'Neill, M., Scanlon, W. & Sezer, S.
01/08/2009 → 30/06/2014
Project: Research