{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-scalability-of-simplicity-empirical","title":"The Scalability of Simplicity: Empirical Analysis of Vision-Language Learning with a Single Transformer","arxiv_id":"2504.10462","date":"2025-04-14","proceeding":null,"authors":["Weixian Lei","Jiacong Wang","Haochen Wang","Xiangtai Li","Jun Hao Liew","Jiashi Feng","Zilong Huang"],"abstract":"This paper introduces SAIL, a single transformer unified multimodal large language model (MLLM) that integrates raw pixel encoding and language decoding within a singular architecture. Unlike existing modular MLLMs, which rely on a pre-trained vision transformer (ViT), SAIL eliminates the need for a separate vision encoder, presenting a more minimalist architecture design. Instead of introducing novel architectural components, SAIL adapts mix-attention mechanisms and multimodal positional encodings to better align with the distinct characteristics of visual and textual modalities. We systematically compare SAIL's properties-including scalability, cross-modal information flow patterns, and visual representation capabilities-with those of modular MLLMs. By scaling both training data and model size, SAIL achieves performance comparable to modular MLLMs. Notably, the removal of pretrained ViT components enhances SAIL's scalability and results in significantly different cross-modal information flow patterns. Moreover, SAIL demonstrates strong visual representation capabilities, achieving results on par with ViT-22B in vision tasks such as semantic segmentation. Code and models are available at https://github.com/bytedance/SAIL.","url_abs":"https://arxiv.org/abs/2504.10462v1","url_pdf":"https://arxiv.org/pdf/2504.10462v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-scalability-of-simplicity-empirical","repo_url":"https://github.com/bytedance/sail","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"large-language-model","task_name":"Large Language Model"},{"task_slug":"multimodal-large-language-model","task_name":"Multimodal Large Language Model"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2504.10462","atlas_url":"https://app.syntology.ai/?focus=2504.10462","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2504.10462"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bytedance/sail","reach":null}],"summary":{"ran":2,"unverified":7},"by_repo_kind":{"official":{"samples":9,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a4e5e73f6afe0623","entry":"MistralMultimodalRotaryEmbedding","repo":"bytedance/sail","repo_kind":"official","path":"modeling_sail.py","file_url":"https://github.com/bytedance/sail/blob/HEAD/modeling_sail.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a4e5e73f6afe0623"}},{"code_sha256_prefix":"6aa4a05b539db200","entry":"SailConfig","repo":"bytedance/sail","repo_kind":"official","path":"modeling_sail.py","file_url":"https://github.com/bytedance/sail/blob/HEAD/modeling_sail.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6aa4a05b539db200"}},{"code_sha256_prefix":"ab536ed99d8736b7","entry":"MistralAttention","repo":"bytedance/sail","repo_kind":"official","path":"modeling_sail.py","file_url":"https://github.com/bytedance/sail/blob/HEAD/modeling_sail.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ab536ed99d8736b7"}},{"code_sha256_prefix":"410f13ace42e01a2","entry":"MistralDecoderLayer","repo":"bytedance/sail","repo_kind":"official","path":"modeling_sail.py","file_url":"https://github.com/bytedance/sail/blob/HEAD/modeling_sail.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"410f13ace42e01a2"}},{"code_sha256_prefix":"73d4ab90a7031b2e","entry":"MistralFlashAttention2","repo":"bytedance/sail","repo_kind":"official","path":"modeling_sail.py","file_url":"https://github.com/bytedance/sail/blob/HEAD/modeling_sail.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"73d4ab90a7031b2e"}},{"code_sha256_prefix":"570379c16d776fcb","entry":"MistralSdpaAttention","repo":"bytedance/sail","repo_kind":"official","path":"modeling_sail.py","file_url":"https://github.com/bytedance/sail/blob/HEAD/modeling_sail.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"570379c16d776fcb"}},{"code_sha256_prefix":"8f502c142a395a08","entry":"MultimodalMistralModel","repo":"bytedance/sail","repo_kind":"official","path":"modeling_sail.py","file_url":"https://github.com/bytedance/sail/blob/HEAD/modeling_sail.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8f502c142a395a08"}},{"code_sha256_prefix":"e36c0055b00afdf8","entry":"SailForCausalLM","repo":"bytedance/sail","repo_kind":"official","path":"modeling_sail.py","file_url":"https://github.com/bytedance/sail/blob/HEAD/modeling_sail.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e36c0055b00afdf8"}},{"code_sha256_prefix":"2c5971642e5ce12f","entry":"apply_multimodal_rotary_pos_emb","repo":"bytedance/sail","repo_kind":"official","path":"modeling_sail.py","file_url":"https://github.com/bytedance/sail/blob/HEAD/modeling_sail.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2c5971642e5ce12f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}