BIGOS (Benchmark Intended Grouping of Open Speech)

The Benchmark Intended Grouping of Open Speech (BIGOS) is a novel corpus specifically designed for Polish Automatic Speech Recognition (ASR) systems. This initial version of the benchmark comprises 1,900 audio recordings from 71 distinct speakers, sourced from 10 publicly available speech corpora¹²³.

Here are some key points about BIGOS:

1. Purpose: BIGOS aims to facilitate systematic benchmarking and tracking of Polish ASR systems over time across a diverse range of publicly available corpora.
2. Evaluation: The benchmark evaluates both proprietary and open-source ASR systems on a diverse set of recordings and their corresponding original transcriptions.
3. Findings:
   • The performance of the latest open-source models is comparable to that of more established commercial services.
   • Model size significantly influences system accuracy.
   • Scenarios involving highly specialized or spontaneous speech show a decrease in accuracy.
4. Challenges: The study discusses the challenges of using public datasets for ASR evaluation and the limitations based on this inaugural benchmark.
5. Availability: The BIGOS corpus and associated tools are publicly available for replication and customization of the benchmark.

In summary, BIGOS provides a valuable resource for advancing Polish ASR research and improving the quality of speech recognition systems in the Polish language¹. Check https://www.semanticscholar.org/paper/BIGOS-Benchmark-Intended-Grouping-of-Open-Speech-Junczyk

Source: Conversation with Bing, 3/18/2024 (1) BIGOS - Benchmark Intended Grouping of Open Speech Corpora for Polish .... https://annals-csis.org/proceedings/2023/pliks/1609.pdf. (2) Annals of Computer Science and Information Systems, Volume 35. https://annals-csis.org/proceedings/2023/drp/1609.html. (3) BIGOS - Benchmark Intended Grouping of Open Speech Corpora for Polish .... https://ieeexplore.ieee.org/abstract/document/10306084.