{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scaling-backwards-minimal-synthetic-pre","title":"Scaling Backwards: Minimal Synthetic Pre-training?","arxiv_id":"2408.00677","date":"2024-08-01","proceeding":null,"authors":["Ryo Nakamura","Ryu Tadokoro","Ryosuke Yamada","Yuki M. Asano","Iro Laina","Christian Rupprecht","Nakamasa Inoue","Rio Yokota","Hirokatsu Kataoka"],"abstract":"Pre-training and transfer learning are an important building block of current computer vision systems. While pre-training is usually performed on large real-world image datasets, in this paper we ask whether this is truly necessary. To this end, we search for a minimal, purely synthetic pre-training dataset that allows us to achieve performance similar to the 1 million images of ImageNet-1k. We construct such a dataset from a single fractal with perturbations. With this, we contribute three main findings. (i) We show that pre-training is effective even with minimal synthetic images, with performance on par with large-scale pre-training datasets like ImageNet-1k for full fine-tuning. (ii) We investigate the single parameter with which we construct artificial categories for our dataset. We find that while the shape differences can be indistinguishable to humans, they are crucial for obtaining strong performances. (iii) Finally, we investigate the minimal requirements for successful pre-training. Surprisingly, we find that a substantial reduction of synthetic images from 1k to 1 can even lead to an increase in pre-training performance, a motivation to further investigate ''scaling backwards''. Finally, we extend our method from synthetic images to real images to see if a single real image can show similar pre-training effect through shape augmentation. We find that the use of grayscale images and affine transformations allows even real images to ''scale backwards''.","url_abs":"https://arxiv.org/abs/2408.00677v2","url_pdf":"https://arxiv.org/pdf/2408.00677v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"scaling-backwards-minimal-synthetic-pre","repo_url":"https://github.com/super-tadory/1p-frac","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2408.00677","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2408.00677"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/super-tadory/1p-frac","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/SUPER-TADORY/1p-frac","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3,"unverified":2},"by_repo_kind":{"official":{"samples":5,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b291fa6a41b0e305","entry":"calc_p","repo":"SUPER-TADORY/1p-frac","repo_kind":"official","path":"1p-frac_generater/src/render_from_oneparam_mp.py","file_url":"https://github.com/SUPER-TADORY/1p-frac/blob/HEAD/1p-frac_generater/src/render_from_oneparam_mp.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b291fa6a41b0e305"}},{"code_sha256_prefix":"6b023b60d9e509b4","entry":"reshape","repo":"SUPER-TADORY/1p-frac","repo_kind":"official","path":"1p-frac_generater/src/render_from_oneparam_mp.py","file_url":"https://github.com/SUPER-TADORY/1p-frac/blob/HEAD/1p-frac_generater/src/render_from_oneparam_mp.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6b023b60d9e509b4"}},{"code_sha256_prefix":"a1c6321806632355","entry":"sample_svs","repo":"SUPER-TADORY/1p-frac","repo_kind":"official","path":"1p-frac_generater/src/utils.py","file_url":"https://github.com/SUPER-TADORY/1p-frac/blob/HEAD/1p-frac_generater/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a1c6321806632355"}},{"code_sha256_prefix":"66801f43fe5c3707","entry":"create_dataloader","repo":"SUPER-TADORY/1p-frac","repo_kind":"official","path":"train/data.py","file_url":"https://github.com/SUPER-TADORY/1p-frac/blob/HEAD/train/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"66801f43fe5c3707"}},{"code_sha256_prefix":"3a41df871027ed4c","entry":"sample_system","repo":"SUPER-TADORY/1p-frac","repo_kind":"official","path":"1p-frac_generater/src/utils.py","file_url":"https://github.com/SUPER-TADORY/1p-frac/blob/HEAD/1p-frac_generater/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3a41df871027ed4c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}