{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/audio-based-ai-classifiers-show-no-evidence","title":"Audio-based AI classifiers show no evidence of improved COVID-19 screening over simple symptoms checkers","arxiv_id":"2212.08570","date":"2022-12-15","proceeding":null,"authors":["Harry Coppock","George Nicholson","Ivan Kiskin","Vasiliki Koutra","Kieran Baker","Jobie Budd","Richard Payne","Emma Karoune","David Hurley","Alexander Titcomb","Sabrina Egglestone","Ana Tendero Cañadas","Lorraine Butler","Radka Jersakova","Jonathon Mellor","Selina Patel","Tracey Thornley","Peter Diggle","Sylvia Richardson","Josef Packham","Björn W. Schuller","Davide Pigoli","Steven Gilmour","Stephen Roberts","Chris Holmes"],"abstract":"Recent work has reported that AI classifiers trained on audio recordings can accurately predict severe acute respiratory syndrome coronavirus 2 (SARSCoV2) infection status. Here, we undertake a large scale study of audio-based deep learning classifiers, as part of the UK governments pandemic response. We collect and analyse a dataset of audio recordings from 67,842 individuals with linked metadata, including reverse transcription polymerase chain reaction (PCR) test outcomes, of whom 23,514 tested positive for SARS CoV 2. Subjects were recruited via the UK governments National Health Service Test-and-Trace programme and the REal-time Assessment of Community Transmission (REACT) randomised surveillance survey. In an unadjusted analysis of our dataset AI classifiers predict SARS-CoV-2 infection status with high accuracy (Receiver Operating Characteristic Area Under the Curve (ROCAUC) 0.846 [0.838, 0.854]) consistent with the findings of previous studies. However, after matching on measured confounders, such as age, gender, and self reported symptoms, our classifiers performance is much weaker (ROC-AUC 0.619 [0.594, 0.644]). Upon quantifying the utility of audio based classifiers in practical settings, we find them to be outperformed by simple predictive scores based on user reported symptoms.","url_abs":"https://arxiv.org/abs/2212.08570v2","url_pdf":"https://arxiv.org/pdf/2212.08570v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"audio-based-ai-classifiers-show-no-evidence","repo_url":"https://github.com/alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":null,"method_name":null},{"method_slug":"test","method_name":"Test"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2212.08570","atlas_url":"https://app.syntology.ai/?focus=2212.08570","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2212.08570"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":2,"ran":5,"unverified":2},"by_repo_kind":{"official":{"samples":9,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2a80220dabcb742a","entry":"conv1x1","repo":"alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers","repo_kind":"official","path":"BNNBaseline/lib/ResNetSource.py","file_url":"https://github.com/alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers/blob/HEAD/BNNBaseline/lib/ResNetSource.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2a80220dabcb742a"}},{"code_sha256_prefix":"600ff2c45e0de056","entry":"conv3x3","repo":"alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers","repo_kind":"official","path":"BNNBaseline/lib/ResNetSource.py","file_url":"https://github.com/alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers/blob/HEAD/BNNBaseline/lib/ResNetSource.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"600ff2c45e0de056"}},{"code_sha256_prefix":"07b5a3d2cc0c0aa2","entry":"evaluate_metrics","repo":"alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers","repo_kind":"official","path":"BNNBaseline/lib/evaluate.py","file_url":"https://github.com/alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers/blob/HEAD/BNNBaseline/lib/evaluate.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"07b5a3d2cc0c0aa2"}},{"code_sha256_prefix":"de1370a2d5e42ab7","entry":"get_file_label_list_for_split","repo":"alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers","repo_kind":"official","path":"BNNBaseline/lib/extract_feat.py","file_url":"https://github.com/alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers/blob/HEAD/BNNBaseline/lib/extract_feat.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de1370a2d5e42ab7"}},{"code_sha256_prefix":"4c75e29bcf8520c2","entry":"isclose","repo":"alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers","repo_kind":"official","path":"data-paper/unit_tests.py","file_url":"https://github.com/alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers/blob/HEAD/data-paper/unit_tests.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4c75e29bcf8520c2"}},{"code_sha256_prefix":"77a09064753e9740","entry":"load_dataset_modality","repo":"alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers","repo_kind":"official","path":"SvmBaseline/combine_features.py","file_url":"https://github.com/alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers/blob/HEAD/SvmBaseline/combine_features.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"77a09064753e9740"}},{"code_sha256_prefix":"c2ec4e1aa5e7224f","entry":"model_predict","repo":"alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers","repo_kind":"official","path":"BNNBaseline/lib/evaluate.py","file_url":"https://github.com/alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers/blob/HEAD/BNNBaseline/lib/evaluate.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c2ec4e1aa5e7224f"}},{"code_sha256_prefix":"72d53618367ce071","entry":"get_file","repo":"alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers","repo_kind":"official","path":"analysis_splits/ciab_data.py","file_url":"https://github.com/alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers/blob/HEAD/analysis_splits/ciab_data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"72d53618367ce071"}},{"code_sha256_prefix":"b35676336da8c1ea","entry":"resnet18","repo":"alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers","repo_kind":"official","path":"BNNBaseline/lib/ResNetSource.py","file_url":"https://github.com/alan-turing-institute/turing-rss-health-data-lab-biomedical-acoustic-markers/blob/HEAD/BNNBaseline/lib/ResNetSource.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b35676336da8c1ea"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}