{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-open-respiratory-acoustic-foundation","title":"Towards Open Respiratory Acoustic Foundation Models: Pretraining and Benchmarking","arxiv_id":"2406.16148","date":"2024-06-23","proceeding":null,"authors":["Yuwei Zhang","Tong Xia","Jing Han","Yu Wu","Georgios Rizos","Yang Liu","Mohammed Mosuily","Jagmohan Chauhan","Cecilia Mascolo"],"abstract":"Respiratory audio, such as coughing and breathing sounds, has predictive power for a wide range of healthcare applications, yet is currently under-explored. The main problem for those applications arises from the difficulty in collecting large labeled task-specific data for model development. Generalizable respiratory acoustic foundation models pretrained with unlabeled data would offer appealing advantages and possibly unlock this impasse. However, given the safety-critical nature of healthcare applications, it is pivotal to also ensure openness and replicability for any proposed foundation model solution. To this end, we introduce OPERA, an OPEn Respiratory Acoustic foundation model pretraining and benchmarking system, as the first approach answering this need. We curate large-scale respiratory audio datasets (~136K samples, over 400 hours), pretrain three pioneering foundation models, and build a benchmark consisting of 19 downstream respiratory health tasks for evaluation. Our pretrained models demonstrate superior performance (against existing acoustic models pretrained with general audio on 16 out of 19 tasks) and generalizability (to unseen datasets and new respiratory audio modalities). This highlights the great promise of respiratory acoustic foundation models and encourages more studies using OPERA as an open resource to accelerate research on respiratory audio for health. The system is accessible from https://github.com/evelyn0414/OPERA.","url_abs":"https://arxiv.org/abs/2406.16148v3","url_pdf":"https://arxiv.org/pdf/2406.16148v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-open-respiratory-acoustic-foundation","repo_url":"https://github.com/evelyn0414/opera","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.16148","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.16148"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/evelyn0414/OPERA","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/evelyn0414/opera","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":6,"ran_draft_wrong":2,"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":10,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"273aded2e209e1ba","entry":"combine_dataloaders","repo":"evelyn0414/OPERA","repo_kind":"official","path":"src/pretrain/cola_training.py","file_url":"https://github.com/evelyn0414/OPERA/blob/HEAD/src/pretrain/cola_training.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"273aded2e209e1ba"}},{"code_sha256_prefix":"ce5db6832cea6b60","entry":"compute_saliency_map","repo":"evelyn0414/OPERA","repo_kind":"official","path":"res_analysis/saliency_map.py","file_url":"https://github.com/evelyn0414/OPERA/blob/HEAD/res_analysis/saliency_map.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ce5db6832cea6b60"}},{"code_sha256_prefix":"1bc472f9c7b0ef4f","entry":"crop_first","repo":"evelyn0414/opera","repo_kind":"official","path":"src/util.py","file_url":"https://github.com/evelyn0414/opera/blob/HEAD/src/util.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1bc472f9c7b0ef4f"}},{"code_sha256_prefix":"119b5fe3e8b4754c","entry":"generate_mask","repo":"evelyn0414/OPERA","repo_kind":"official","path":"res_analysis/visualize_masked_spec.py","file_url":"https://github.com/evelyn0414/OPERA/blob/HEAD/res_analysis/visualize_masked_spec.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"119b5fe3e8b4754c"}},{"code_sha256_prefix":"e50a9c988c664528","entry":"random_crop","repo":"evelyn0414/opera","repo_kind":"official","path":"src/util.py","file_url":"https://github.com/evelyn0414/opera/blob/HEAD/src/util.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e50a9c988c664528"}},{"code_sha256_prefix":"8ceb445ccf7af790","entry":"random_mask","repo":"evelyn0414/opera","repo_kind":"official","path":"src/util.py","file_url":"https://github.com/evelyn0414/opera/blob/HEAD/src/util.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8ceb445ccf7af790"}},{"code_sha256_prefix":"1629c61058c17fa6","entry":"random_masking","repo":"evelyn0414/OPERA","repo_kind":"official","path":"res_analysis/visualize_masked_spec.py","file_url":"https://github.com/evelyn0414/OPERA/blob/HEAD/res_analysis/visualize_masked_spec.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1629c61058c17fa6"}},{"code_sha256_prefix":"d33a98b078a5f213","entry":"replace_generation","repo":"evelyn0414/OPERA","repo_kind":"official","path":"res_analysis/visualize_masked_spec.py","file_url":"https://github.com/evelyn0414/OPERA/blob/HEAD/res_analysis/visualize_masked_spec.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d33a98b078a5f213"}},{"code_sha256_prefix":"0416d4fe99ed3480","entry":"smooth_list","repo":"evelyn0414/OPERA","repo_kind":"official","path":"res_analysis/show_logs.py","file_url":"https://github.com/evelyn0414/OPERA/blob/HEAD/res_analysis/show_logs.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0416d4fe99ed3480"}},{"code_sha256_prefix":"5ee822af0adca359","entry":"get_encoder_path","repo":"evelyn0414/OPERA","repo_kind":"official","path":"src/benchmark/model_util.py","file_url":"https://github.com/evelyn0414/OPERA/blob/HEAD/src/benchmark/model_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5ee822af0adca359"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}