{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2606-00579","title":"Sandboxed Coding Agents are Competitive Omni-modal Task Solvers","arxiv_id":"2606.00579","date":"2026-05-30","proceeding":null,"authors":["Dongping Chen","Xuanao Huang","Zhihan Hu","Qingyuan Shi","Dianqi Li","Tianyi Zhou"],"abstract":"As multimodal LLMs increasingly target video and audio, it is often assumed that such tasks require native omnimodal models. We show that this is not always the case: coding agents with only text+image access and a sandboxed tool-use interface can match, and in several settings outperform, SOTA native omnimodal models and predefined multimodal agent scaffolds across multiple audio-video benchmarks. Our trajectory analysis suggests that their strength comes from writing code and orchestrating tools to extract relevant evidence from transcripts, frames, and other modality signals, thereby converting omnimodal tasks into retrieval and information-processing problems rather than ingesting entire media streams. We further characterize their limitations through a failure taxonomy and process-level trace analysis, and show that simple skill injection, including human-written and self-distilled skills, substantially improves performance. To explore open-source elicitation, we introduce Code-X, a training recipe with the OmniCoding trajectory dataset and verifiable reward, and provide baselines on Qwen-3.5-9B and Qwen-3.6-27B. Finally, we argue that the next frontier is many-modality processing, and introduce TerminalBench-O, a process-level benchmark for real-world omnimodal processing tasks. Code will be available at https://github.com/Dongping-Chen/OmniCoding.","url_abs":"https://arxiv.org/abs/2606.00579","url_pdf":"https://arxiv.org/pdf/2606.00579","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2606.00579"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/Dongping-Chen/OmniCoding","reach":null}],"summary":{"ran":8,"unverified":4},"by_repo_kind":{"found_in_text":{"samples":12,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":12,"samples":[{"code_sha256_prefix":"635cabd9c6473475","entry":"decode_subcall_images_to_dir","repo":"Dongping-Chen/OmniCoding","repo_kind":"found_in_text","path":"src/omnicoding/harnesses/kira_io.py","file_url":"https://github.com/Dongping-Chen/OmniCoding/blob/HEAD/src/omnicoding/harnesses/kira_io.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"635cabd9c6473475"}},{"code_sha256_prefix":"51e9b7e8d7ad73f8","entry":"extract_answer","repo":"Dongping-Chen/OmniCoding","repo_kind":"found_in_text","path":"src/omnicoding/rl/reward.py","file_url":"https://github.com/Dongping-Chen/OmniCoding/blob/HEAD/src/omnicoding/rl/reward.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"51e9b7e8d7ad73f8"}},{"code_sha256_prefix":"f473a70b4d5702b0","entry":"extract_answer_from_messages","repo":"Dongping-Chen/OmniCoding","repo_kind":"found_in_text","path":"src/omnicoding/rl/reward.py","file_url":"https://github.com/Dongping-Chen/OmniCoding/blob/HEAD/src/omnicoding/rl/reward.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f473a70b4d5702b0"}},{"code_sha256_prefix":"3fe9f2401bd6c555","entry":"filter_by_source_indices","repo":"Dongping-Chen/OmniCoding","repo_kind":"found_in_text","path":"src/omnicoding/harnesses/kira_io.py","file_url":"https://github.com/Dongping-Chen/OmniCoding/blob/HEAD/src/omnicoding/harnesses/kira_io.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"3fe9f2401bd6c555"}},{"code_sha256_prefix":"5995d2ce911da0dc","entry":"normalize","repo":"Dongping-Chen/OmniCoding","repo_kind":"found_in_text","path":"src/omnicoding/rl/reward.py","file_url":"https://github.com/Dongping-Chen/OmniCoding/blob/HEAD/src/omnicoding/rl/reward.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"5995d2ce911da0dc"}},{"code_sha256_prefix":"ad935d94e2c3ab80","entry":"prepare_subset","repo":"Dongping-Chen/OmniCoding","repo_kind":"found_in_text","path":"src/omnicoding/benchmarks/subsets.py","file_url":"https://github.com/Dongping-Chen/OmniCoding/blob/HEAD/src/omnicoding/benchmarks/subsets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"ad935d94e2c3ab80"}},{"code_sha256_prefix":"09abe599f437dda7","entry":"sha256_bytes","repo":"Dongping-Chen/OmniCoding","repo_kind":"found_in_text","path":"src/omnicoding/benchmarks/subsets.py","file_url":"https://github.com/Dongping-Chen/OmniCoding/blob/HEAD/src/omnicoding/benchmarks/subsets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"09abe599f437dda7"}},{"code_sha256_prefix":"5789280e1f024de4","entry":"sha256_file","repo":"Dongping-Chen/OmniCoding","repo_kind":"found_in_text","path":"src/omnicoding/benchmarks/subsets.py","file_url":"https://github.com/Dongping-Chen/OmniCoding/blob/HEAD/src/omnicoding/benchmarks/subsets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"5789280e1f024de4"}},{"code_sha256_prefix":"ad42d7afda418131","entry":"can_escalate","repo":"Dongping-Chen/OmniCoding","repo_kind":"found_in_text","path":"src/omnicoding/harnesses/effort_policy.py","file_url":"https://github.com/Dongping-Chen/OmniCoding/blob/HEAD/src/omnicoding/harnesses/effort_policy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"ad42d7afda418131"}},{"code_sha256_prefix":"90c11597a7c38f09","entry":"main","repo":"Dongping-Chen/OmniCoding","repo_kind":"found_in_text","path":"src/omnicoding/rl/run_trajectory.py","file_url":"https://github.com/Dongping-Chen/OmniCoding/blob/HEAD/src/omnicoding/rl/run_trajectory.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"90c11597a7c38f09"}},{"code_sha256_prefix":"15e855873853b533","entry":"setup_per_job_venv","repo":"Dongping-Chen/OmniCoding","repo_kind":"found_in_text","path":"src/omnicoding/harnesses/kira_io.py","file_url":"https://github.com/Dongping-Chen/OmniCoding/blob/HEAD/src/omnicoding/harnesses/kira_io.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"15e855873853b533"}},{"code_sha256_prefix":"315b9fde53cf3126","entry":"step_limit_for_effort","repo":"Dongping-Chen/OmniCoding","repo_kind":"found_in_text","path":"src/omnicoding/harnesses/effort_policy.py","file_url":"https://github.com/Dongping-Chen/OmniCoding/blob/HEAD/src/omnicoding/harnesses/effort_policy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"315b9fde53cf3126"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}