Publication Details
Title: The Blame Game in Meeting Room ASR: An Analysis of Feature versus Model Errors in Noisy and Mismatched Conditions
Author: S. H. Krishnan Parthasarathi, S.-Y. Chang, J. Cohen, N. Morgan, and S. Wegmann
Bibliographic Information: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2013), Vancouver, Canada
Date: May 2013
Research Area: Speech
Type: Article in conference proceedings
PDF: https://www.icsi.berkeley.edu/pubs/speech/blamegame13.pdf
Overview:
Given a test waveform, state-of-the-art ASR systems extract a sequence of MFCC features and decode them with a set of trained HMMs. When this test data is clean, and it matches the condition used for training the models, then there are few errors. While it is known that ASR systems are brittle in noisy or mismatched conditions, there has been little work in quantitatively attributing the errors to features or to models. This paper attributes the sources of these errors in three conditions: (a) matched near-field, (b) matched far-field, and a (c) mismatched condition. We undertake a series of diagnostic analyses employing the bootstrap method to probe a meeting room ASR system. Results show that when the conditions are matched (even if they are far-field), the model errors dominate; however, in mismatched conditions features are neither invariant nor separable and this causes as many errors as the model does.
Acknowledgements:
This work was partially supported by funding provided to ICSI by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA) through the project "What’s wrong with ASR, and how can we fix it." All statements of fact, opinion or conclusions contained herein are those of the authors and should not be construed as representing the official views or policies of the IARPA, the ODNI or the U.S. Government.
Bibliographic Reference:
S. H. Krishnan Parthasarathi, S.-Y. Chang, J. Cohen, N. Morgan, and S. Wegmann. The Blame Game in Meeting Room ASR: An Analysis of Feature versus Model Errors in Noisy and Mismatched Conditions. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2013), Vancouver, Canada, May 2013
Author: S. H. Krishnan Parthasarathi, S.-Y. Chang, J. Cohen, N. Morgan, and S. Wegmann
Bibliographic Information: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2013), Vancouver, Canada
Date: May 2013
Research Area: Speech
Type: Article in conference proceedings
PDF: https://www.icsi.berkeley.edu/pubs/speech/blamegame13.pdf
Overview:
Given a test waveform, state-of-the-art ASR systems extract a sequence of MFCC features and decode them with a set of trained HMMs. When this test data is clean, and it matches the condition used for training the models, then there are few errors. While it is known that ASR systems are brittle in noisy or mismatched conditions, there has been little work in quantitatively attributing the errors to features or to models. This paper attributes the sources of these errors in three conditions: (a) matched near-field, (b) matched far-field, and a (c) mismatched condition. We undertake a series of diagnostic analyses employing the bootstrap method to probe a meeting room ASR system. Results show that when the conditions are matched (even if they are far-field), the model errors dominate; however, in mismatched conditions features are neither invariant nor separable and this causes as many errors as the model does.
Acknowledgements:
This work was partially supported by funding provided to ICSI by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA) through the project "What’s wrong with ASR, and how can we fix it." All statements of fact, opinion or conclusions contained herein are those of the authors and should not be construed as representing the official views or policies of the IARPA, the ODNI or the U.S. Government.
Bibliographic Reference:
S. H. Krishnan Parthasarathi, S.-Y. Chang, J. Cohen, N. Morgan, and S. Wegmann. The Blame Game in Meeting Room ASR: An Analysis of Feature versus Model Errors in Noisy and Mismatched Conditions. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2013), Vancouver, Canada, May 2013