{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/foresight-deep-generative-modelling-of","title":"Foresight -- Generative Pretrained Transformer (GPT) for Modelling of Patient Timelines using EHRs","arxiv_id":"2212.08072","date":"2022-12-13","proceeding":null,"authors":["Zeljko Kraljevic","Dan Bean","Anthony Shek","Rebecca Bendayan","Harry Hemingway","Joshua Au Yeung","Alexander Deng","Alfie Baston","Jack Ross","Esther Idowu","James T Teo","Richard J Dobson"],"abstract":"Background: Electronic Health Records hold detailed longitudinal information about each patient's health status and general clinical history, a large portion of which is stored within the unstructured text. Existing approaches focus mostly on structured data and a subset of single-domain outcomes. We explore how temporal modelling of patients from free text and structured data, using deep generative transformers can be used to forecast a wide range of future disorders, substances, procedures or findings. Methods: We present Foresight, a novel transformer-based pipeline that uses named entity recognition and linking tools to convert document text into structured, coded concepts, followed by providing probabilistic forecasts for future medical events such as disorders, substances, procedures and findings. We processed the entire free-text portion from three different hospital datasets totalling 811336 patients covering both physical and mental health. Findings: On tests in two UK hospitals (King's College Hospital, South London and Maudsley) and the US MIMIC-III dataset precision@10 0.68, 0.76 and 0.88 was achieved for forecasting the next disorder in a patient timeline, while precision@10 of 0.80, 0.81 and 0.91 was achieved for forecasting the next biomedical concept. Foresight was also validated on 34 synthetic patient timelines by five clinicians and achieved relevancy of 97% for the top forecasted candidate disorder. As a generative model, it can forecast follow-on biomedical concepts for as many steps as required. Interpretation: Foresight is a general-purpose model for biomedical concept modelling that can be used for real-world risk forecasting, virtual trials and clinical research to study the progression of disorders, simulate interventions and counterfactuals, and educational purposes.","url_abs":"https://arxiv.org/abs/2212.08072v2","url_pdf":"https://arxiv.org/pdf/2212.08072v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"foresight-deep-generative-modelling-of","repo_url":"https://github.com/cogstack/foresight","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"foresight-deep-generative-modelling-of","repo_url":"https://github.com/CogStack/MedCAT","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"cg","task_name":"NER"},{"task_slug":"named-entity-recognition-1","task_name":"Named Entity Recognition"},{"task_slug":"named-entity-recognition-ner","task_name":"Named Entity Recognition (NER)"},{"task_slug":"named-entity-recognition","task_name":"named-entity-recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2212.08072","atlas_url":"https://app.syntology.ai/?focus=2212.08072","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2212.08072"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/CogStack/MedCAT","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cogstack/foresight","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":12},"by_repo_kind":{"official":{"samples":12,"ran":12,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ac14695298be58fc","entry":"calculate_counts","repo":"cogstack/foresight","repo_kind":"official","path":"foresight/utils/stream_utils.py","file_url":"https://github.com/cogstack/foresight/blob/HEAD/foresight/utils/stream_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ac14695298be58fc"}},{"code_sha256_prefix":"35e52a8c531602c4","entry":"docs2stream","repo":"cogstack/foresight","repo_kind":"official","path":"foresight/utils/stream_utils.py","file_url":"https://github.com/cogstack/foresight/blob/HEAD/foresight/utils/stream_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"35e52a8c531602c4"}},{"code_sha256_prefix":"ac9a04968e7acc5d","entry":"filter_by_type","repo":"cogstack/foresight","repo_kind":"official","path":"foresight/datasets/filters.py","file_url":"https://github.com/cogstack/foresight/blob/HEAD/foresight/datasets/filters.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ac9a04968e7acc5d"}},{"code_sha256_prefix":"f7578191b70a44ca","entry":"get_all_splits","repo":"cogstack/foresight","repo_kind":"official","path":"foresight/datasets/utils.py","file_url":"https://github.com/cogstack/foresight/blob/HEAD/foresight/datasets/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f7578191b70a44ca"}},{"code_sha256_prefix":"2c5b98657b286418","entry":"get_children_map","repo":"cogstack/foresight","repo_kind":"official","path":"foresight/utils/cdb_utils.py","file_url":"https://github.com/cogstack/foresight/blob/HEAD/foresight/utils/cdb_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2c5b98657b286418"}},{"code_sha256_prefix":"15764b4be0c5f12f","entry":"get_duration_separator","repo":"cogstack/foresight","repo_kind":"official","path":"foresight/datasets/utils.py","file_url":"https://github.com/cogstack/foresight/blob/HEAD/foresight/datasets/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"15764b4be0c5f12f"}},{"code_sha256_prefix":"3249e2235dcbb16b","entry":"get_entities_for_doc","repo":"cogstack/foresight","repo_kind":"official","path":"foresight/utils/stream_utils.py","file_url":"https://github.com/cogstack/foresight/blob/HEAD/foresight/utils/stream_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3249e2235dcbb16b"}},{"code_sha256_prefix":"996a28aa4ca8328e","entry":"get_parents_map","repo":"cogstack/foresight","repo_kind":"official","path":"foresight/utils/cdb_utils.py","file_url":"https://github.com/cogstack/foresight/blob/HEAD/foresight/utils/cdb_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"996a28aa4ca8328e"}},{"code_sha256_prefix":"7ef0f00604eab68d","entry":"make_example","repo":"cogstack/foresight","repo_kind":"official","path":"foresight/datasets/utils.py","file_url":"https://github.com/cogstack/foresight/blob/HEAD/foresight/datasets/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7ef0f00604eab68d"}},{"code_sha256_prefix":"6b31e0ea8a1bb7e8","entry":"metrics_data2df","repo":"cogstack/foresight","repo_kind":"official","path":"foresight/metrics/next_concept_prediction.py","file_url":"https://github.com/cogstack/foresight/blob/HEAD/foresight/metrics/next_concept_prediction.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6b31e0ea8a1bb7e8"}},{"code_sha256_prefix":"bcd0c6ffe7251ad8","entry":"precision_on_one","repo":"cogstack/foresight","repo_kind":"official","path":"foresight/metrics/next_concept_prediction.py","file_url":"https://github.com/cogstack/foresight/blob/HEAD/foresight/metrics/next_concept_prediction.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bcd0c6ffe7251ad8"}},{"code_sha256_prefix":"e0f388f78415fc22","entry":"reverse_pt2ch","repo":"cogstack/foresight","repo_kind":"official","path":"foresight/utils/cdb_utils.py","file_url":"https://github.com/cogstack/foresight/blob/HEAD/foresight/utils/cdb_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e0f388f78415fc22"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}