{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mimic-iv-icd-a-new-benchmark-for-extreme-1","title":"Mimic-IV-ICD: A new benchmark for eXtreme MultiLabel Classification","arxiv_id":"2304.13998","date":"2023-04-27","proceeding":null,"authors":["Thanh-Tung Nguyen","Viktor Schlegel","Abhinav Kashyap","Stefan Winkler","Shao-Syuan Huang","Jie-Jyun Liu","Chih-Jen Lin"],"abstract":"Clinical notes are assigned ICD codes - sets of codes for diagnoses and procedures. In the recent years, predictive machine learning models have been built for automatic ICD coding. However, there is a lack of widely accepted benchmarks for automated ICD coding models based on large-scale public EHR data. This paper proposes a public benchmark suite for ICD-10 coding using a large EHR dataset derived from MIMIC-IV, the most recent public EHR dataset. We implement and compare several popular methods for ICD coding prediction tasks to standardize data preprocessing and establish a comprehensive ICD coding benchmark dataset. This approach fosters reproducibility and model comparison, accelerating progress toward employing automated ICD coding in future studies. Furthermore, we create a new ICD-9 benchmark using MIMIC-IV data, providing more data points and a higher number of ICD codes than MIMIC-III. Our open-source code offers easy access to data processing steps, benchmark creation, and experiment replication for those with MIMIC-IV access, providing insights, guidance, and protocols to efficiently develop ICD coding models.","url_abs":"https://arxiv.org/abs/2304.13998v1","url_pdf":"https://arxiv.org/pdf/2304.13998v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mimic-iv-icd-a-new-benchmark-for-extreme-1","repo_url":"https://github.com/thomasnguyen92/MIMIC-IV-ICD-data-processing","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"multi-label-classification","task_name":"Multi-Label Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd-10-1","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD-10-full","model":"MSMN","rank_in_archive_order":1,"of":5,"metrics":{"Macro-AUC":"97.07","Macro-F1":"5.42","Micro-AUC":"99.61","Micro-F1":"55.91","Precision@8":"67.66"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd-10-1","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD-10-full","model":"Joint LAAT","rank_in_archive_order":2,"of":5,"metrics":{"Macro-AUC":"93.64","Macro-F1":"5.71","Micro-AUC":"99.27","Micro-F1":"55.89","Precision@8":"66.89"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd-10-1","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD-10-full","model":"LAAT","rank_in_archive_order":3,"of":5,"metrics":{"Macro-AUC":"92.96","Macro-F1":"4.47","Micro-AUC":"99.14","Micro-F1":"55.40","Precision@8":"66,97"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd-10-1","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD-10-full","model":"PLM","rank_in_archive_order":4,"of":5,"metrics":{"Macro-AUC":"91.85","Macro-F1":"4.90","Micro-AUC":"99.02","Micro-F1":"56.95","Precision@8":"69.47"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd-10-1","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD-10-full","model":"CAML","rank_in_archive_order":5,"of":5,"metrics":{"Macro-AUC":"89.91","Macro-F1":"4.07","Micro-AUC":"98.79","Micro-F1":"52.67","Precision@8":"64.43"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd10","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD10-top50","model":"MSMN","rank_in_archive_order":1,"of":5,"metrics":{"AUC (Macro)":"93.60","AUC (Micro)":"95.61","F1 (macro)":"69.01","F1 (micro)":"74.15","Precision@5":"65.16"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd10","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD10-top50","model":"PLM-ICD","rank_in_archive_order":2,"of":5,"metrics":{"AUC (Macro)":"93.37","AUC (Micro)":"95.69","F1 (macro)":"70.31 ","F1 (micro)":"73.27","Precision@5":"64.57"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd10","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD10-top50","model":"Joint LAAT","rank_in_archive_order":3,"of":5,"metrics":{"AUC (Macro)":"93.39","AUC (Micro)":"95.57","F1 (macro)":"68.41","F1 (micro)":"72.85","Precision@5":"64.49"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd10","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD10-top50","model":"LAAT","rank_in_archive_order":4,"of":5,"metrics":{"AUC (Macro)":"93.21","AUC (Micro)":"95.49","F1 (macro)":"68.15","F1 (micro)":"72.56","Precision@5":"64.39"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd10","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD10-top50","model":"CAML","rank_in_archive_order":5,"of":5,"metrics":{"AUC (Macro)":"91.05","AUC (Micro)":"93.18","F1 (macro)":"64.30","F1 (micro)":"67.56","Precision@5":"59.58"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd9-full","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD9-full","model":"MSMN","rank_in_archive_order":1,"of":5,"metrics":{"F1 Macro":"13.94","F1 Micro":"61.15","Macro AUC":"96.79","Micro AUC":"99.56","Precision@8":"68.89"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd9-full","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD9-full","model":"PLM-ICD","rank_in_archive_order":2,"of":5,"metrics":{"F1 Macro":"14.40","F1 Micro":"62.45","Macro AUC":"96.61","Micro AUC":"99.53","Precision@8":"70.34"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd9-full","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD9-full","model":"Joint LAAT","rank_in_archive_order":3,"of":5,"metrics":{"F1 Macro":"14.17","F1 Micro":"60.37","Macro AUC":"95.57","Micro AUC":"99.49","Precision@8":"67.46"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd9-full","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD9-full","model":"LAAT","rank_in_archive_order":4,"of":5,"metrics":{"F1 Macro":"13.12","F1 Micro":"60.31","Macro AUC":"95.18","Micro AUC":"99.47","Precision@8":"67.47"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd9-full","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD9-full","model":"CAML","rank_in_archive_order":5,"of":5,"metrics":{"F1 Macro":"11.06","F1 Micro":"57.28","Macro AUC":"93.45","Micro AUC":"99.29","Precision@8":"64.91"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd9","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD9-top50","model":"MSMN","rank_in_archive_order":1,"of":5,"metrics":{"AUC Macro":"95.13","AUC Micro":"96.46","F1 Macro":"71.85","F1 Micro":"75.78","Precision @5":"62.60"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd9","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD9-top50","model":"PLM-ICD","rank_in_archive_order":2,"of":5,"metrics":{"AUC Macro":"94.97","AUC Micro":"96.41","F1 Macro":"71.35","F1 Micro":"75.46","Precision @5":"62.44"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd9","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD9-top50","model":"Joint LAAT","rank_in_archive_order":3,"of":5,"metrics":{"AUC Macro":"94.92","AUC Micro":"96.31","F1 Macro":"69.93","F1 Micro":"74.33","Precision @5":"61.95"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd9","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD9-top50","model":"LAAT","rank_in_archive_order":4,"of":5,"metrics":{"AUC Macro":"94.88","AUC Micro":"96.29","F1 Macro":"69.99","F1 Micro":"74.46","Precision @5":"62.01"},"uses_additional_data":false},{"leaderboard":"/sota/medical-code-prediction-on-mimic-iv-icd9","task":"Medical Code Prediction","dataset":"MIMIC-IV-ICD9-top50","model":"CAML","rank_in_archive_order":5,"of":5,"metrics":{"AUC Macro":"93.07","AUC Micro":"94.05","F1 Macro":"65.33","F1 Micro":"69.23","Precision @5":"58.64"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}