{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generatect-text-guided-3d-chest-ct-generation","title":"GenerateCT: Text-Conditional Generation of 3D Chest CT Volumes","arxiv_id":"2305.16037","date":"2023-05-25","proceeding":null,"authors":["Ibrahim Ethem Hamamci","Sezgin Er","Anjany Sekuboyina","Enis Simsar","Alperen Tezcan","Ayse Gulnihan Simsek","Sevval Nil Esirgun","Furkan Almas","Irem Dogan","Muhammed Furkan Dasdelen","Chinmay Prabhakar","Hadrien Reynaud","Sarthak Pati","Christian Bluethgen","Mehmet Kemal Ozdemir","Bjoern Menze"],"abstract":"GenerateCT, the first approach to generating 3D medical imaging conditioned on free-form medical text prompts, incorporates a text encoder and three key components: a novel causal vision transformer for encoding 3D CT volumes, a text-image transformer for aligning CT and text tokens, and a text-conditional super-resolution diffusion model. Without directly comparable methods in 3D medical imaging, we benchmarked GenerateCT against cutting-edge methods, demonstrating its superiority across all key metrics. Importantly, we evaluated GenerateCT's clinical applications in a multi-abnormality classification task. First, we established a baseline by training a multi-abnormality classifier on our real dataset. To further assess the model's generalization to external data and performance with unseen prompts in a zero-shot scenario, we employed an external set to train the classifier, setting an additional benchmark. We conducted two experiments in which we doubled the training datasets by synthesizing an equal number of volumes for each set using GenerateCT. The first experiment demonstrated an 11% improvement in the AP score when training the classifier jointly on real and generated volumes. The second experiment showed a 7% improvement when training on both real and generated volumes based on unseen prompts. Moreover, GenerateCT enables the scaling of synthetic training datasets to arbitrary sizes. As an example, we generated 100,000 3D CTs, fivefold the number in our real set, and trained the classifier exclusively on these synthetic CTs. Impressively, this classifier surpassed the performance of the one trained on all available real data by a margin of 8%. Last, domain experts evaluated the generated volumes, confirming a high degree of alignment with the text prompt. Access our code, model weights, training data, and generated data at https://github.com/ibrahimethemhamamci/GenerateCT","url_abs":"https://arxiv.org/abs/2305.16037v5","url_pdf":"https://arxiv.org/pdf/2305.16037v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generatect-text-guided-3d-chest-ct-generation","repo_url":"https://github.com/ibrahimethemhamamci/generatect","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"computed-tomography-ct","task_name":"Computed Tomography (CT)"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"large-language-model","task_name":"Large Language Model"},{"task_slug":"medical-image-generation","task_name":"Medical Image Generation"},{"task_slug":"super-resolution","task_name":"Super-Resolution"},{"task_slug":"synthetic-data-generation","task_name":"Synthetic Data Generation"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2305.16037","atlas_url":"https://app.syntology.ai/?focus=2305.16037","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.16037"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ibrahimethemhamamci/generatect","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_violates":4,"ran_honours":1,"ran_draft_wrong":1,"ran":4,"unverified":5},"by_repo_kind":{"official":{"samples":15,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"856d86b39c6005b4","entry":"divisible_by","repo":"ibrahimethemhamamci/generatect","repo_kind":"official","path":"super_resolution/super_resolution/superres_pytorch.py","file_url":"https://github.com/ibrahimethemhamamci/generatect/blob/HEAD/super_resolution/super_resolution/superres_pytorch.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":2,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"856d86b39c6005b4"}},{"code_sha256_prefix":"7f1040f5e3991d5e","entry":"identity","repo":"ibrahimethemhamamci/generatect","repo_kind":"official","path":"super_resolution/super_resolution/superres_pytorch.py","file_url":"https://github.com/ibrahimethemhamamci/generatect/blob/HEAD/super_resolution/super_resolution/superres_pytorch.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":2,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7f1040f5e3991d5e"}},{"code_sha256_prefix":"b5472673332de78e","entry":"log","repo":"ibrahimethemhamamci/generatect","repo_kind":"official","path":"super_resolution/super_resolution/elucidated_superres.py","file_url":"https://github.com/ibrahimethemhamamci/generatect/blob/HEAD/super_resolution/super_resolution/elucidated_superres.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b5472673332de78e"}},{"code_sha256_prefix":"fc8652b9a9069275","entry":"ListOrTuple","repo":"ibrahimethemhamamci/generatect","repo_kind":"official","path":"super_resolution/super_resolution/configs.py","file_url":"https://github.com/ibrahimethemhamamci/generatect/blob/HEAD/super_resolution/super_resolution/configs.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fc8652b9a9069275"}},{"code_sha256_prefix":"efcd80d66cfe5bcb","entry":"cast_tuple","repo":"ibrahimethemhamamci/generatect","repo_kind":"official","path":"super_resolution/super_resolution/trainer.py","file_url":"https://github.com/ibrahimethemhamamci/generatect/blob/HEAD/super_resolution/super_resolution/trainer.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"efcd80d66cfe5bcb"}},{"code_sha256_prefix":"a47706b35198fab3","entry":"convert_image_to","repo":"ibrahimethemhamamci/generatect","repo_kind":"official","path":"super_resolution/super_resolution/data.py","file_url":"https://github.com/ibrahimethemhamamci/generatect/blob/HEAD/super_resolution/super_resolution/data.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a47706b35198fab3"}},{"code_sha256_prefix":"60fff7c3c400d7ff","entry":"default","repo":"ibrahimethemhamamci/generatect","repo_kind":"official","path":"super_resolution/super_resolution/configs.py","file_url":"https://github.com/ibrahimethemhamamci/generatect/blob/HEAD/super_resolution/super_resolution/configs.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"60fff7c3c400d7ff"}},{"code_sha256_prefix":"915fbeb88e13d4a3","entry":"default","repo":"ibrahimethemhamamci/generatect","repo_kind":"official","path":"super_resolution/super_resolution/t5.py","file_url":"https://github.com/ibrahimethemhamamci/generatect/blob/HEAD/super_resolution/super_resolution/t5.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"915fbeb88e13d4a3"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"ibrahimethemhamamci/generatect","repo_kind":"official","path":"super_resolution/super_resolution/configs.py","file_url":"https://github.com/ibrahimethemhamamci/generatect/blob/HEAD/super_resolution/super_resolution/configs.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"0ba8f745cf192018","entry":"get_images_dataloader","repo":"ibrahimethemhamamci/generatect","repo_kind":"official","path":"super_resolution/super_resolution/data.py","file_url":"https://github.com/ibrahimethemhamamci/generatect/blob/HEAD/super_resolution/super_resolution/data.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0ba8f745cf192018"}},{"code_sha256_prefix":"bba86ad0e88a5402","entry":"delay2str","repo":"ibrahimethemhamamci/generatect","repo_kind":"official","path":"inference_superres.py","file_url":"https://github.com/ibrahimethemhamamci/generatect/blob/HEAD/inference_superres.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bba86ad0e88a5402"}},{"code_sha256_prefix":"89fb6a86f187a44f","entry":"get_exp_name","repo":"ibrahimethemhamamci/generatect","repo_kind":"official","path":"inference_superres.py","file_url":"https://github.com/ibrahimethemhamamci/generatect/blob/HEAD/inference_superres.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"89fb6a86f187a44f"}},{"code_sha256_prefix":"4af30ae5e0b7d6ca","entry":"get_tokenizer","repo":"ibrahimethemhamamci/generatect","repo_kind":"official","path":"super_resolution/super_resolution/t5.py","file_url":"https://github.com/ibrahimethemhamamci/generatect/blob/HEAD/super_resolution/super_resolution/t5.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4af30ae5e0b7d6ca"}},{"code_sha256_prefix":"b39b6369a080033e","entry":"scan_folder","repo":"ibrahimethemhamamci/generatect","repo_kind":"official","path":"inference_transformer.py","file_url":"https://github.com/ibrahimethemhamamci/generatect/blob/HEAD/inference_transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b39b6369a080033e"}},{"code_sha256_prefix":"c25b827ebea33670","entry":"update_config_with_arg","repo":"ibrahimethemhamamci/generatect","repo_kind":"official","path":"inference_superres.py","file_url":"https://github.com/ibrahimethemhamamci/generatect/blob/HEAD/inference_superres.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c25b827ebea33670"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}