{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/serab-a-multi-lingual-benchmark-for-speech","title":"SERAB: A multi-lingual benchmark for speech emotion recognition","arxiv_id":"2110.03414","date":"2021-10-07","proceeding":null,"authors":["Neil Scheidwasser-Clow","Mikolaj Kegler","Pierre Beckmann","Milos Cernak"],"abstract":"Recent developments in speech emotion recognition (SER) often leverage deep neural networks (DNNs). Comparing and benchmarking different DNN models can often be tedious due to the use of different datasets and evaluation protocols. To facilitate the process, here, we present the Speech Emotion Recognition Adaptation Benchmark (SERAB), a framework for evaluating the performance and generalization capacity of different approaches for utterance-level SER. The benchmark is composed of nine datasets for SER in six languages. Since the datasets have different sizes and numbers of emotional classes, the proposed setup is particularly suitable for estimating the generalization capacity of pre-trained DNN-based feature extractors. We used the proposed framework to evaluate a selection of standard hand-crafted feature sets and state-of-the-art DNN representations. The results highlight that using only a subset of the data included in SERAB can result in biased evaluation, while compliance with the proposed protocol can circumvent this issue.","url_abs":"https://arxiv.org/abs/2110.03414v1","url_pdf":"https://arxiv.org/pdf/2110.03414v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"serab-a-multi-lingual-benchmark-for-speech","repo_url":"https://github.com/neclow/serab","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"serab-a-multi-lingual-benchmark-for-speech","repo_url":"https://github.com/gasserelbanna/serab-byols","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"emotion-recognition","task_name":"Emotion Recognition"},{"task_slug":"speech-emotion-recognition","task_name":"Speech Emotion Recognition"}],"methods":[{"method_slug":null,"method_name":"qb Experts Team"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2110.03414","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.03414"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/neclow/serab","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gasserelbanna/serab-byols","reach":null}],"summary":{"ran_violates":1,"ran_honours":1,"ran_draft_wrong":3,"unverified":4},"by_repo_kind":{"official":{"samples":9,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1c0f269ae9673b08","entry":"default","repo":"neclow/serab","repo_kind":"official","path":"SERAB/byol_a/byol_pytorch.py","file_url":"https://github.com/neclow/serab/blob/HEAD/SERAB/byol_a/byol_pytorch.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1c0f269ae9673b08"}},{"code_sha256_prefix":"4daa3e876f494d75","entry":"flatten","repo":"neclow/serab","repo_kind":"official","path":"SERAB/byol_a/byol_pytorch.py","file_url":"https://github.com/neclow/serab/blob/HEAD/SERAB/byol_a/byol_pytorch.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4daa3e876f494d75"}},{"code_sha256_prefix":"ca4faa49da53bd5c","entry":"group_by_key_prefix_and_remove_prefix","repo":"neclow/serab","repo_kind":"official","path":"SERAB/byol_a/models/cvt.py","file_url":"https://github.com/neclow/serab/blob/HEAD/SERAB/byol_a/models/cvt.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ca4faa49da53bd5c"}},{"code_sha256_prefix":"afcf8f2954675d55","entry":"group_dict_by_key","repo":"neclow/serab","repo_kind":"official","path":"SERAB/byol_a/models/cvt.py","file_url":"https://github.com/neclow/serab/blob/HEAD/SERAB/byol_a/models/cvt.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"afcf8f2954675d55"}},{"code_sha256_prefix":"c6bc247ffa36099c","entry":"singleton","repo":"neclow/serab","repo_kind":"official","path":"SERAB/byol_a/byol_pytorch.py","file_url":"https://github.com/neclow/serab/blob/HEAD/SERAB/byol_a/byol_pytorch.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c6bc247ffa36099c"}},{"code_sha256_prefix":"43df9c4c776065ab","entry":"filter_label","repo":"neclow/serab","repo_kind":"official","path":"tensorflow_datasets/iem4/iem4.py","file_url":"https://github.com/neclow/serab/blob/HEAD/tensorflow_datasets/iem4/iem4.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"43df9c4c776065ab"}},{"code_sha256_prefix":"4d6a494114a246c0","entry":"parse_name","repo":"neclow/serab","repo_kind":"official","path":"tensorflow_datasets/aesdd/aesdd.py","file_url":"https://github.com/neclow/serab/blob/HEAD/tensorflow_datasets/aesdd/aesdd.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4d6a494114a246c0"}},{"code_sha256_prefix":"deca7d5b2b1aedbf","entry":"parse_speaker_id","repo":"neclow/serab","repo_kind":"official","path":"tensorflow_datasets/aesdd/aesdd.py","file_url":"https://github.com/neclow/serab/blob/HEAD/tensorflow_datasets/aesdd/aesdd.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"deca7d5b2b1aedbf"}},{"code_sha256_prefix":"20c4b4b697ce1d05","entry":"parse_txt_file","repo":"neclow/serab","repo_kind":"official","path":"tensorflow_datasets/iem4/iem4.py","file_url":"https://github.com/neclow/serab/blob/HEAD/tensorflow_datasets/iem4/iem4.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"20c4b4b697ce1d05"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}