{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2608-07713","title":"Tokenizer Generator Coupling in Medical Image Generation","arxiv_id":"2608.07713","date":"2026-08-07","proceeding":null,"authors":["Liam Chalcroft"],"abstract":"Latent medical image generators usually treat the tokenizer as fixed preprocessing. We test whether this separation is valid in a controlled ChestMNIST study at 64x64, crossing discrete tokenizers, generator families, and sampler settings under a shared latent grid, with continuous-latent reference cells. In this controlled setting, rankings depend jointly on the tokenizer, generator, and sampler: the best quantizer changes with the generator, and validation-based sampler selection changes the apparent generator ranking. We retrain the vocabulary-1024 interaction block at three seeds and the interaction survives (6 of 9 pairwise quantizer comparisons exceed three seed standard deviations), and we scope the wider single-seed grid accordingly. Reconstruction PSNR alone is not a reliable selection criterion; we instead introduce a generator-free statistic, neighbour-conditional predictive gain, that separates the quantizer families by downstream generation quality (rank-AUC 1.00) where reconstruction PSNR and marginal token entropy do not. On LFQ-1024, retuning D3PM and SE-D3PM (selected on a held-out validation split) moves them from default FID-192 0.44/0.41 to 0.09/0.10 at lower NFE, replicated across seeds; the continuous references were not given an equivalent sampler sweep. We report FID-192 as an internal ranking metric; it ranks consistently with standard FID-2048 (Spearman 0.80) and with a label-free classifier two-sample test (0.78). We interpret these results through a rate-distortion-modelability framing, where modelability is conditional on the generator, sampler, and inference budget. All experiments are at 64x64 on low-resolution medical-style images, unconditional, and evaluated with non-clinical FID-based metrics, and we scope every claim to that setting. Code: https://github.com/liamchalcroft/medtokenizers and https://github.com/liamchalcroft/medlatents.","url_abs":"https://arxiv.org/abs/2608.07713","url_pdf":"https://arxiv.org/pdf/2608.07713","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2608.07713","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2608.07713"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/liamchalcroft/medtokenizers","reach":null},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/liamchalcroft/medlatents","reach":null}],"summary":{"ran":14,"unverified":6},"by_repo_kind":{"found_in_text":{"samples":20,"ran":14,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2af91ede38d69da0","entry":"collate_conditional_batch","repo":"liamchalcroft/medlatents","repo_kind":"found_in_text","path":"src/medlatents/conditioning/collate.py","file_url":"https://github.com/liamchalcroft/medlatents/blob/HEAD/src/medlatents/conditioning/collate.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2af91ede38d69da0"}},{"code_sha256_prefix":"5c2e304ef7c3d032","entry":"compute_mae","repo":"liamchalcroft/medtokenizers","repo_kind":"found_in_text","path":"src/medtokenizers/evaluation/metrics.py","file_url":"https://github.com/liamchalcroft/medtokenizers/blob/HEAD/src/medtokenizers/evaluation/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5c2e304ef7c3d032"}},{"code_sha256_prefix":"4b806df4ec7208fb","entry":"compute_mse","repo":"liamchalcroft/medtokenizers","repo_kind":"found_in_text","path":"src/medtokenizers/evaluation/metrics.py","file_url":"https://github.com/liamchalcroft/medtokenizers/blob/HEAD/src/medtokenizers/evaluation/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4b806df4ec7208fb"}},{"code_sha256_prefix":"b6ddbbeb8cebe8e7","entry":"compute_psnr","repo":"liamchalcroft/medtokenizers","repo_kind":"found_in_text","path":"src/medtokenizers/evaluation/metrics.py","file_url":"https://github.com/liamchalcroft/medtokenizers/blob/HEAD/src/medtokenizers/evaluation/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b6ddbbeb8cebe8e7"}},{"code_sha256_prefix":"fe74f7596bc3cd62","entry":"create_conditional_collate_fn","repo":"liamchalcroft/medlatents","repo_kind":"found_in_text","path":"src/medlatents/conditioning/collate.py","file_url":"https://github.com/liamchalcroft/medlatents/blob/HEAD/src/medlatents/conditioning/collate.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe74f7596bc3cd62"}},{"code_sha256_prefix":"a87d6af7f04a9527","entry":"create_model_variants","repo":"liamchalcroft/medlatents","repo_kind":"found_in_text","path":"src/medlatents/configs.py","file_url":"https://github.com/liamchalcroft/medlatents/blob/HEAD/src/medlatents/configs.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a87d6af7f04a9527"}},{"code_sha256_prefix":"eaa2e30c02052d7b","entry":"get_conv","repo":"liamchalcroft/medtokenizers","repo_kind":"found_in_text","path":"src/medtokenizers/modules/layers.py","file_url":"https://github.com/liamchalcroft/medtokenizers/blob/HEAD/src/medtokenizers/modules/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eaa2e30c02052d7b"}},{"code_sha256_prefix":"6d05388476add5d7","entry":"get_cosine_schedule_with_warmup","repo":"liamchalcroft/medlatents","repo_kind":"found_in_text","path":"src/medlatents/common/schedules.py","file_url":"https://github.com/liamchalcroft/medlatents/blob/HEAD/src/medlatents/common/schedules.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6d05388476add5d7"}},{"code_sha256_prefix":"b10e8e79b1c89c97","entry":"get_padding","repo":"liamchalcroft/medtokenizers","repo_kind":"found_in_text","path":"src/medtokenizers/modules/layers.py","file_url":"https://github.com/liamchalcroft/medtokenizers/blob/HEAD/src/medtokenizers/modules/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b10e8e79b1c89c97"}},{"code_sha256_prefix":"3464031d3269e008","entry":"jaxtyped_compile_safe","repo":"liamchalcroft/medtokenizers","repo_kind":"found_in_text","path":"src/medtokenizers/modules/utils.py","file_url":"https://github.com/liamchalcroft/medtokenizers/blob/HEAD/src/medtokenizers/modules/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3464031d3269e008"}},{"code_sha256_prefix":"cb632039824220ae","entry":"load_latents","repo":"liamchalcroft/medtokenizers","repo_kind":"found_in_text","path":"src/medtokenizers/inference.py","file_url":"https://github.com/liamchalcroft/medtokenizers/blob/HEAD/src/medtokenizers/inference.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb632039824220ae"}},{"code_sha256_prefix":"d97836d8c1ecc9ee","entry":"pad_divisible","repo":"liamchalcroft/medtokenizers","repo_kind":"found_in_text","path":"src/medtokenizers/preprocessing.py","file_url":"https://github.com/liamchalcroft/medtokenizers/blob/HEAD/src/medtokenizers/preprocessing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d97836d8c1ecc9ee"}},{"code_sha256_prefix":"834d158f1320c065","entry":"percentile_normalize","repo":"liamchalcroft/medtokenizers","repo_kind":"found_in_text","path":"src/medtokenizers/preprocessing.py","file_url":"https://github.com/liamchalcroft/medtokenizers/blob/HEAD/src/medtokenizers/preprocessing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"834d158f1320c065"}},{"code_sha256_prefix":"7061c151f4be243f","entry":"resample_to_spacing","repo":"liamchalcroft/medtokenizers","repo_kind":"found_in_text","path":"src/medtokenizers/preprocessing.py","file_url":"https://github.com/liamchalcroft/medtokenizers/blob/HEAD/src/medtokenizers/preprocessing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7061c151f4be243f"}},{"code_sha256_prefix":"614326f7bc65a4c4","entry":"create_encoder","repo":"liamchalcroft/medlatents","repo_kind":"found_in_text","path":"src/medlatents/conditioning/encoders.py","file_url":"https://github.com/liamchalcroft/medlatents/blob/HEAD/src/medlatents/conditioning/encoders.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"614326f7bc65a4c4"}},{"code_sha256_prefix":"35851a6f82840307","entry":"get_config","repo":"liamchalcroft/medtokenizers","repo_kind":"found_in_text","path":"src/medtokenizers/config.py","file_url":"https://github.com/liamchalcroft/medtokenizers/blob/HEAD/src/medtokenizers/config.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"35851a6f82840307"}},{"code_sha256_prefix":"eb41b5ce332542b3","entry":"load_indices","repo":"liamchalcroft/medtokenizers","repo_kind":"found_in_text","path":"src/medtokenizers/inference.py","file_url":"https://github.com/liamchalcroft/medtokenizers/blob/HEAD/src/medtokenizers/inference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eb41b5ce332542b3"}},{"code_sha256_prefix":"de2375ed3a91d92c","entry":"sample_timesteps_u_shaped","repo":"liamchalcroft/medlatents","repo_kind":"found_in_text","path":"src/medlatents/common/time.py","file_url":"https://github.com/liamchalcroft/medlatents/blob/HEAD/src/medlatents/common/time.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de2375ed3a91d92c"}},{"code_sha256_prefix":"c0b00bf63adf9872","entry":"sample_timesteps_uniform","repo":"liamchalcroft/medlatents","repo_kind":"found_in_text","path":"src/medlatents/common/time.py","file_url":"https://github.com/liamchalcroft/medlatents/blob/HEAD/src/medlatents/common/time.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c0b00bf63adf9872"}},{"code_sha256_prefix":"78dfddf27211614b","entry":"sample_timesteps_uniform_shifted","repo":"liamchalcroft/medlatents","repo_kind":"found_in_text","path":"src/medlatents/common/time.py","file_url":"https://github.com/liamchalcroft/medlatents/blob/HEAD/src/medlatents/common/time.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"78dfddf27211614b"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}