{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2601-07773","title":"Self-transcendence: Is External Feature Guidance Indispensable for Accelerating Diffusion Transformer Training?","arxiv_id":"2601.07773","date":"2026-01-12","proceeding":null,"authors":["Lingchen Sun","Rongyuan Wu","Zhengqiang Zhang","Ruibin Li","Yujing Sun","Shuaizheng Liu","Lei Zhang"],"abstract":"Recent works such as REPA have shown that guiding diffusion models with external semantic features (e.g., DINO) can significantly accelerate the training of diffusion transformers (DiTs). However, the use of pretrained external features as guidance signals introduces additional dependencies. We argue that DiTs actually have the power to guide the training of themselves, and propose SelfTranscendence, an effective method that achieves fast convergence using internal feature supervision only. The desired internal guidance features should meet two requirements: structurally clean to help shallow blocks separate noise from signal, and semantically discriminative to help shallow layers learn effective representations. With this consideration, we first align the DiT features with the clean VAE latent features, a native component of latent diffusion, for a short training phase (e.g., 40 epochs) to improve their structural representations, then apply the classifier-free guidance to the intermediate features, enhancing their discriminative capability and semantic expressiveness. These enriched internal features, learned entirely within the model, are used as supervision signals to guide a new DiT training from scratch. Compared to existing self-contained methods, our approach achieves a significant performance boost. It can even surpass REPA, which uses the external DINO features as guidance, in both generation quality and convergence speed for both class-to-image and text-to-image generation tasks. The source code of our method can be found at https://github.com/csslc/Self-Transcendence.","url_abs":"https://arxiv.org/abs/2601.07773","url_pdf":"https://arxiv.org/pdf/2601.07773","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2601.07773","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2601.07773"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/csslc/Self-Transcendence","reach":null}],"summary":{"ran":3,"ran_draft_wrong":1,"ran_honours":1,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":6,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"ce261fdb436234fb","entry":"FinalLayer","repo":"csslc/Self-Transcendence","repo_kind":"found_in_text","path":"models/sit_selftrans_model.py","file_url":"https://github.com/csslc/Self-Transcendence/blob/HEAD/models/sit_selftrans_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ce261fdb436234fb"}},{"code_sha256_prefix":"4fd1214124625591","entry":"SiTBlock","repo":"csslc/Self-Transcendence","repo_kind":"found_in_text","path":"models/sit_selftrans_model.py","file_url":"https://github.com/csslc/Self-Transcendence/blob/HEAD/models/sit_selftrans_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4fd1214124625591"}},{"code_sha256_prefix":"85d8dab86a801978","entry":"TimestepEmbedder","repo":"csslc/Self-Transcendence","repo_kind":"found_in_text","path":"models/sit_selftrans_model.py","file_url":"https://github.com/csslc/Self-Transcendence/blob/HEAD/models/sit_selftrans_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"85d8dab86a801978"}},{"code_sha256_prefix":"b05a447cc3b9e991","entry":"build_mlp","repo":"csslc/Self-Transcendence","repo_kind":"found_in_text","path":"models/sit_selftrans_model.py","file_url":"https://github.com/csslc/Self-Transcendence/blob/HEAD/models/sit_selftrans_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b05a447cc3b9e991"}},{"code_sha256_prefix":"1df7f271ba4dbf2a","entry":"get_2d_sincos_pos_embed","repo":"csslc/Self-Transcendence","repo_kind":"found_in_text","path":"models/sit_selftrans_model.py","file_url":"https://github.com/csslc/Self-Transcendence/blob/HEAD/models/sit_selftrans_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1df7f271ba4dbf2a"}},{"code_sha256_prefix":"a0273b1105ea2880","entry":"SiT","repo":"csslc/Self-Transcendence","repo_kind":"found_in_text","path":"models/sit_selftrans_model.py","file_url":"https://github.com/csslc/Self-Transcendence/blob/HEAD/models/sit_selftrans_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a0273b1105ea2880"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CV","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}