Earnings-22

Earnings-22 is a practical benchmark designed to evaluate automatic speech recognition (ASR) systems' performance on real-world, accented audio. Let me provide you with more details:

1. Corpus Description:
2. Earnings-22 consists of 125 audio files totaling 119 hours of English-language earnings calls. These calls were gathered from global companies.

3. Unlike many existing corpora, Earnings-22 focuses on speech in the wild, representing real-world scenarios where accents and environmental conditions vary.

4. Purpose and Significance:

5. ASR systems have achieved impressive performance on common corpora but often struggle with real-world speech.
6. Earnings-22 aims to bridge this gap by providing a free-to-use benchmark that includes accented audio.

7. Researchers and industry professionals can use Earnings-22 to evaluate and improve ASR models' robustness.

8. Comparison and Insights:

9. The benchmark involves four commercial ASR models, and their performance is compared.
10. By considering the country of origin, the study reveals variations in ASR accuracy.

11. Individual Word Error Rate (IWER) analysis highlights how certain accents impact model performance more than others.

12. Academic and Industrial Impact:

13. Earnings-22 serves as a valuable resource for both academic research and industrial applications.
14. It provides a realistic dataset for evaluating ASR systems' effectiveness in handling diverse accents.

Source: Conversation with Bing, 3/16/2024 (1) Earnings-22: A Practical Benchmark for Accents in the Wild. https://arxiv.org/abs/2203.15591. (2) Earnings-22: A Practical Benchmark for Accents in the Wild. https://deepai.org/publication/earnings-22-a-practical-benchmark-for-accents-in-the-wild. (3) arXiv:2203.15591v1 [cs.CL] 29 Mar 2022. https://arxiv.org/pdf/2203.15591.pdf. (4) undefined. https://doi.org/10.48550/arXiv.2203.15591.