{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multi-omics-prediction-from-high-content","title":"Multi-omics Prediction from High-content Cellular Imaging with Deep Learning","arxiv_id":"2306.09391","date":"2023-06-15","proceeding":null,"authors":["Rahil Mehrizi","Arash Mehrjou","Maryana Alegro","Yi Zhao","Benedetta Carbone","Carl Fishwick","Johanna Vappiani","Jing Bi","Siobhan Sanford","Hakan Keles","Marcus Bantscheff","Cuong Nguyen","Patrick Schwab"],"abstract":"High-content cellular imaging, transcriptomics, and proteomics data provide rich and complementary views on the molecular layers of biology that influence cellular states and function. However, the biological determinants through which changes in multi-omics measurements influence cellular morphology have not yet been systematically explored, and the degree to which cell imaging could potentially enable the prediction of multi-omics directly from cell imaging data is therefore currently unclear. Here, we address the question of whether it is possible to predict bulk multi-omics measurements directly from cell images using Image2Omics - a deep learning approach that predicts multi-omics in a cell population directly from high-content images of cells stained with multiplexed fluorescent dyes. We perform an experimental evaluation in gene-edited macrophages derived from human induced pluripotent stem cells (hiPSC) under multiple stimulation conditions and demonstrate that Image2Omics achieves significantly better performance in predicting transcriptomics and proteomics measurements directly from cell images than predictions based on the mean observed training set abundance. We observed significant predictability of abundances for 4927 (18.72%; 95% CI: 6.52%, 35.52%) and 3521 (13.38%; 95% CI: 4.10%, 32.21%) transcripts out of 26137 in M1 and M2-stimulated macrophages respectively and for 422 (8.46%; 95% CI: 0.58%, 25.83%) and 697 (13.98%; 95% CI: 2.41%, 32.83%) proteins out of 4986 in M1 and M2-stimulated macrophages respectively. Our results show that some transcript and protein abundances are predictable from cell imaging and that cell imaging may potentially, in some settings and depending on the mechanisms of interest and desired performance threshold, even be a scalable and resource-efficient substitute for multi-omics measurements.","url_abs":"https://arxiv.org/abs/2306.09391v3","url_pdf":"https://arxiv.org/pdf/2306.09391v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multi-omics-prediction-from-high-content","repo_url":"https://github.com/gsk-ai/image2omics","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2306.09391","atlas_url":"https://app.syntology.ai/?focus=2306.09391","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.09391"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gsk-ai/image2omics","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":8},"by_repo_kind":{"official":{"samples":8,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"63b42428dde1ae49","entry":"aggregate_array","repo":"gsk-ai/image2omics","repo_kind":"official","path":"image2omics/aggregate.py","file_url":"https://github.com/gsk-ai/image2omics/blob/HEAD/image2omics/aggregate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"63b42428dde1ae49"}},{"code_sha256_prefix":"06a272c95d33edd8","entry":"aggregate_df","repo":"gsk-ai/image2omics","repo_kind":"official","path":"image2omics/aggregate.py","file_url":"https://github.com/gsk-ai/image2omics/blob/HEAD/image2omics/aggregate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"06a272c95d33edd8"}},{"code_sha256_prefix":"b09e1570c8f74e54","entry":"aggregate_df_pooled_variance","repo":"gsk-ai/image2omics","repo_kind":"official","path":"image2omics/aggregate.py","file_url":"https://github.com/gsk-ai/image2omics/blob/HEAD/image2omics/aggregate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b09e1570c8f74e54"}},{"code_sha256_prefix":"0b8471b75ee88abb","entry":"get_aggregation_layer","repo":"gsk-ai/image2omics","repo_kind":"official","path":"image2omics/models/aggregation.py","file_url":"https://github.com/gsk-ai/image2omics/blob/HEAD/image2omics/models/aggregation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0b8471b75ee88abb"}},{"code_sha256_prefix":"1e934edaa9e748ba","entry":"get_dataloader","repo":"gsk-ai/image2omics","repo_kind":"official","path":"image2omics/dataloader.py","file_url":"https://github.com/gsk-ai/image2omics/blob/HEAD/image2omics/dataloader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1e934edaa9e748ba"}},{"code_sha256_prefix":"89732272c6eaff29","entry":"import_from","repo":"gsk-ai/image2omics","repo_kind":"official","path":"image2omics/import.py","file_url":"https://github.com/gsk-ai/image2omics/blob/HEAD/image2omics/import.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"89732272c6eaff29"}},{"code_sha256_prefix":"2f103edb34554969","entry":"smape","repo":"gsk-ai/image2omics","repo_kind":"official","path":"image2omics/plotting.py","file_url":"https://github.com/gsk-ai/image2omics/blob/HEAD/image2omics/plotting.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2f103edb34554969"}},{"code_sha256_prefix":"dbce08979f9dbd30","entry":"split_train_val","repo":"gsk-ai/image2omics","repo_kind":"official","path":"image2omics/dataset_utils.py","file_url":"https://github.com/gsk-ai/image2omics/blob/HEAD/image2omics/dataset_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dbce08979f9dbd30"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}