{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fit-flexible-vision-transformer-for-diffusion","title":"FiT: Flexible Vision Transformer for Diffusion Model","arxiv_id":"2402.12376","date":"2024-02-19","proceeding":null,"authors":["Zeyu Lu","Zidong Wang","Di Huang","Chengyue Wu","Xihui Liu","Wanli Ouyang","Lei Bai"],"abstract":"Nature is infinitely resolution-free. In the context of this reality, existing diffusion models, such as Diffusion Transformers, often face challenges when processing image resolutions outside of their trained domain. To overcome this limitation, we present the Flexible Vision Transformer (FiT), a transformer architecture specifically designed for generating images with unrestricted resolutions and aspect ratios. Unlike traditional methods that perceive images as static-resolution grids, FiT conceptualizes images as sequences of dynamically-sized tokens. This perspective enables a flexible training strategy that effortlessly adapts to diverse aspect ratios during both training and inference phases, thus promoting resolution generalization and eliminating biases induced by image cropping. Enhanced by a meticulously adjusted network structure and the integration of training-free extrapolation techniques, FiT exhibits remarkable flexibility in resolution extrapolation generation. Comprehensive experiments demonstrate the exceptional performance of FiT across a broad range of resolutions, showcasing its effectiveness both within and beyond its training resolution distribution. Repository available at https://github.com/whlzy/FiT.","url_abs":"https://arxiv.org/abs/2402.12376v4","url_pdf":"https://arxiv.org/pdf/2402.12376v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fit-flexible-vision-transformer-for-diffusion","repo_url":"https://github.com/whlzy/fit","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"fit-flexible-vision-transformer-for-diffusion","repo_url":"https://github.com/mindspore-lab/mindone","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}}],"tasks":[{"task_slug":"computational-efficiency","task_name":"Computational Efficiency"},{"task_slug":"image-cropping","task_name":"Image Cropping"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"model","task_name":"model"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"speed","method_name":"SPEED"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2402.12376","atlas_url":"https://app.syntology.ai/?focus=2402.12376","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.12376"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/whlzy/fit","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mindspore-lab/mindone","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}}],"summary":{"ran_draft_wrong":4,"ran":10,"ran_honours":1,"unverified":2},"by_repo_kind":{"official":{"samples":17,"ran":15,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d6a68e210556f857","entry":"approx_standard_normal_cdf","repo":"whlzy/fit","repo_kind":"official","path":"fit/scheduler/improved_diffusion/diffusion_utils.py","file_url":"https://github.com/whlzy/fit/blob/HEAD/fit/scheduler/improved_diffusion/diffusion_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d6a68e210556f857"}},{"code_sha256_prefix":"ab1c9568b4e13899","entry":"continuous_gaussian_log_likelihood","repo":"whlzy/fit","repo_kind":"official","path":"fit/scheduler/improved_diffusion/diffusion_utils.py","file_url":"https://github.com/whlzy/fit/blob/HEAD/fit/scheduler/improved_diffusion/diffusion_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ab1c9568b4e13899"}},{"code_sha256_prefix":"85ccfb32256956d8","entry":"create_norm","repo":"whlzy/fit","repo_kind":"official","path":"fit/model/norms.py","file_url":"https://github.com/whlzy/fit/blob/HEAD/fit/model/norms.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"85ccfb32256956d8"}},{"code_sha256_prefix":"7a6ba78116641182","entry":"create_npz_from_sample_folder","repo":"whlzy/fit","repo_kind":"official","path":"fit/utils/eval_utils.py","file_url":"https://github.com/whlzy/fit/blob/HEAD/fit/utils/eval_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7a6ba78116641182"}},{"code_sha256_prefix":"191c81a96b555436","entry":"find_correction_factor","repo":"whlzy/fit","repo_kind":"official","path":"fit/model/rope.py","file_url":"https://github.com/whlzy/fit/blob/HEAD/fit/model/rope.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"191c81a96b555436"}},{"code_sha256_prefix":"b4a38f39f9288869","entry":"find_correction_range","repo":"whlzy/fit","repo_kind":"official","path":"fit/model/rope.py","file_url":"https://github.com/whlzy/fit/blob/HEAD/fit/model/rope.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b4a38f39f9288869"}},{"code_sha256_prefix":"a8366f238618edad","entry":"get_constant_schedule","repo":"whlzy/fit","repo_kind":"official","path":"fit/utils/lr_scheduler.py","file_url":"https://github.com/whlzy/fit/blob/HEAD/fit/utils/lr_scheduler.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a8366f238618edad"}},{"code_sha256_prefix":"5e2efd2c5b6288c6","entry":"get_constant_schedule_with_warmup","repo":"whlzy/fit","repo_kind":"official","path":"fit/utils/lr_scheduler.py","file_url":"https://github.com/whlzy/fit/blob/HEAD/fit/utils/lr_scheduler.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5e2efd2c5b6288c6"}},{"code_sha256_prefix":"30d11b7b4428ab6e","entry":"get_obj_from_str","repo":"whlzy/fit","repo_kind":"official","path":"fit/utils/utils.py","file_url":"https://github.com/whlzy/fit/blob/HEAD/fit/utils/utils.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"30d11b7b4428ab6e"}},{"code_sha256_prefix":"1d8274f9142363e1","entry":"get_parameter_dtype","repo":"whlzy/fit","repo_kind":"official","path":"fit/model/utils.py","file_url":"https://github.com/whlzy/fit/blob/HEAD/fit/model/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1d8274f9142363e1"}},{"code_sha256_prefix":"9cc5235bd8b13556","entry":"get_piecewise_constant_schedule","repo":"whlzy/fit","repo_kind":"official","path":"fit/utils/lr_scheduler.py","file_url":"https://github.com/whlzy/fit/blob/HEAD/fit/utils/lr_scheduler.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9cc5235bd8b13556"}},{"code_sha256_prefix":"18d8fb19d766c90f","entry":"instantiate_from_config","repo":"whlzy/fit","repo_kind":"official","path":"fit/utils/utils.py","file_url":"https://github.com/whlzy/fit/blob/HEAD/fit/utils/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"18d8fb19d766c90f"}},{"code_sha256_prefix":"5034871c1d2783cf","entry":"linear_ramp_mask","repo":"whlzy/fit","repo_kind":"official","path":"fit/model/rope.py","file_url":"https://github.com/whlzy/fit/blob/HEAD/fit/model/rope.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5034871c1d2783cf"}},{"code_sha256_prefix":"03310bba324ae4fb","entry":"modulate","repo":"whlzy/fit","repo_kind":"official","path":"fit/model/utils.py","file_url":"https://github.com/whlzy/fit/blob/HEAD/fit/model/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"03310bba324ae4fb"}},{"code_sha256_prefix":"8afbfc42c6ea0448","entry":"normal_kl","repo":"whlzy/fit","repo_kind":"official","path":"fit/scheduler/improved_diffusion/diffusion_utils.py","file_url":"https://github.com/whlzy/fit/blob/HEAD/fit/scheduler/improved_diffusion/diffusion_utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8afbfc42c6ea0448"}},{"code_sha256_prefix":"f573c816626189bc","entry":"default","repo":"whlzy/fit","repo_kind":"official","path":"fit/utils/utils.py","file_url":"https://github.com/whlzy/fit/blob/HEAD/fit/utils/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f573c816626189bc"}},{"code_sha256_prefix":"2fc6fdb85f13dc41","entry":"none_or_str","repo":"whlzy/fit","repo_kind":"official","path":"fit/utils/sit_eval_utils.py","file_url":"https://github.com/whlzy/fit/blob/HEAD/fit/utils/sit_eval_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2fc6fdb85f13dc41"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}