{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/aria-an-open-multimodal-native-mixture-of","title":"Aria: An Open Multimodal Native Mixture-of-Experts Model","arxiv_id":"2410.05993","date":"2024-10-08","proceeding":null,"authors":["Dongxu Li","Yudong Liu","HaoNing Wu","Yue Wang","Zhiqi Shen","Bowen Qu","Xinyao Niu","Fan Zhou","Chengen Huang","Yanpeng Li","Chongyan Zhu","Xiaoyi Ren","Chao Li","Yifan Ye","Peng Liu","Lihuan Zhang","Hanshu Yan","Guoyin Wang","Bei Chen","Junnan Li"],"abstract":"Information comes in diverse modalities. Multimodal native AI models are essential to integrate real-world information and deliver comprehensive understanding. While proprietary multimodal native models exist, their lack of openness imposes obstacles for adoptions, let alone adaptations. To fill this gap, we introduce Aria, an open multimodal native model with best-in-class performance across a wide range of multimodal, language, and coding tasks. Aria is a mixture-of-expert model with 3.9B and 3.5B activated parameters per visual token and text token, respectively. It outperforms Pixtral-12B and Llama3.2-11B, and is competitive against the best proprietary models on various multimodal tasks. We pre-train Aria from scratch following a 4-stage pipeline, which progressively equips the model with strong capabilities in language understanding, multimodal understanding, long context window, and instruction following. We open-source the model weights along with a codebase that facilitates easy adoptions and adaptations of Aria in real-world applications.","url_abs":"https://arxiv.org/abs/2410.05993v4","url_pdf":"https://arxiv.org/pdf/2410.05993v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"aria-an-open-multimodal-native-mixture-of","repo_url":"https://github.com/rhymes-ai/aria","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"instruction-following","task_name":"Instruction Following"},{"task_slug":"mixture-of-experts","task_name":"Mixture-of-Experts"},{"task_slug":"video-question-answering","task_name":"Video Question Answering"}],"methods":[{"method_slug":"aria","method_name":"ARiA"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-question-answering-on-tvbench","task":"Video Question Answering","dataset":"TVBench","model":"Aria","rank_in_archive_order":11,"of":28,"metrics":{"Average Accuracy":"51.0"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.05993","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.05993"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rhymes-ai/aria","reach":null}],"summary":{"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"97b470ccfab26be1","entry":"build_aria_instruction","repo":"rhymes-ai/aria","repo_kind":"official","path":"examples/code_sft/inference.py","file_url":"https://github.com/rhymes-ai/aria/blob/HEAD/examples/code_sft/inference.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"97b470ccfab26be1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}