{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2609-14857","title":"ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement","arxiv_id":"2609.14857","date":"2026-09-14","proceeding":null,"authors":["Siwei Wu","Jincheng Ren","Yizhi Li","Haau-Sing Li","Chengran Yang","Yuxuan Zhang","Weicheng Gu","Jian Yang","Riza Batista-Navarro","Chuanyi Zhang","Xianglong Liu","Ming Zhou","Bryan Dai","Chenghua Lin"],"abstract":"Recent work extends recursive self-improvement (RSI) to agent harnesses for long-horizon coding and terminal tasks, enabling agents to improve execution mechanisms from experience. However, generalizable harness RSI remains challenging. First, evolving harnesses on evaluation benchmarks or their subsets makes it difficult to distinguish reusable improvements from benchmark-specific adaptation. Second, single-trajectory updates can conflate systematic harness deficiencies with instance-specific reasoning and solution details, producing modifications that transfer poorly to unseen tasks. Third, localizing recurring behavioral deficiencies within monolithic harnesses is difficult, while whole-harness optimization can entangle unrelated mechanisms and complicate attribution and validation. We propose ModularRSI, a benchmark-disjoint, contrastive, and modular framework for generalizable harness evolution. ModularRSI contrasts successful and failed trajectories for the same task and aggregates evidence across tasks to identify recurring behavioral deficiencies. It decomposes the evolvable harness into five functional modules: Agent Loop, Tool Use, Observation Management, Context Management, and Task Completion Detection. Each module evolves independently within a restricted modification scope, followed by an integration stage that combines the evolved modules into a unified harness and resolves potential conflicts. To support benchmark-disjoint evolution, we curate 2,000 executable evolution tasks from external sources that are disjoint from downstream evaluation benchmarks. Experiments on TB2.0 and SWE-Bench Verified show consistent improvements on unseen in-domain and cross-domain tasks, with the evolved harness also transferring across different foundation models.","url_abs":"https://arxiv.org/abs/2609.14857","url_pdf":"https://arxiv.org/pdf/2609.14857","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2609.14857","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2609.14857"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/IQuestLab/ModularRSI","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":3,"unverified":3},"by_repo_kind":{"found_in_text":{"samples":6,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"de19ee7afd87e58d","entry":"compare_numeric","repo":"IQuestLab/ModularRSI","repo_kind":"found_in_text","path":"adapters/dabstep/scorer.py","file_url":"https://github.com/IQuestLab/ModularRSI/blob/HEAD/adapters/dabstep/scorer.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"de19ee7afd87e58d"}},{"code_sha256_prefix":"5b8eb1533af10264","entry":"extract_numeric","repo":"IQuestLab/ModularRSI","repo_kind":"found_in_text","path":"adapters/dabstep/scorer.py","file_url":"https://github.com/IQuestLab/ModularRSI/blob/HEAD/adapters/dabstep/scorer.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5b8eb1533af10264"}},{"code_sha256_prefix":"a62495edc6f98b22","entry":"is_numeric_with_commas","repo":"IQuestLab/ModularRSI","repo_kind":"found_in_text","path":"adapters/dabstep/scorer.py","file_url":"https://github.com/IQuestLab/ModularRSI/blob/HEAD/adapters/dabstep/scorer.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a62495edc6f98b22"}},{"code_sha256_prefix":"62f29b2074bb2f0a","entry":"get_task_category","repo":"IQuestLab/ModularRSI","repo_kind":"found_in_text","path":"adapters/bfcl/generate_parity_sample.py","file_url":"https://github.com/IQuestLab/ModularRSI/blob/HEAD/adapters/bfcl/generate_parity_sample.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"62f29b2074bb2f0a"}},{"code_sha256_prefix":"8ee53879e7ca7e6c","entry":"load_task_ids_from_bfcl","repo":"IQuestLab/ModularRSI","repo_kind":"found_in_text","path":"adapters/bfcl/generate_parity_sample.py","file_url":"https://github.com/IQuestLab/ModularRSI/blob/HEAD/adapters/bfcl/generate_parity_sample.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8ee53879e7ca7e6c"}},{"code_sha256_prefix":"4b1bef41af5fc1d1","entry":"stratified_sample","repo":"IQuestLab/ModularRSI","repo_kind":"found_in_text","path":"adapters/bfcl/generate_parity_sample.py","file_url":"https://github.com/IQuestLab/ModularRSI/blob/HEAD/adapters/bfcl/generate_parity_sample.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4b1bef41af5fc1d1"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_backlog_21d_20260915.json"},"syntology_extracted_results":{"kind":"leaderboard_placements","source":"Syntology's leaderboard-shaped extractor over the paper's own arXiv-HTML tables: a model pointed at a cell, the number was read from that cell and checked against the board's metric, dataset, split and scale, and an independent check accepted the entry; not reviewed by the paper's authors or the archive's editors","extractor_model":"global.anthropic.claude-sonnet-4-5-20250929-v1:0","verifier_model":null,"prompt_sha":"fa63d4bb9d755694","coverage":{"sentence":"Syntology has checked 6,264 of the 9,581 papers on this site that are newer than the archive; results from the others appear after they are checked.","papers_newer_than_archive":9581,"papers_checked":6264},"entries":[],"not_placed":{"boards":0,"rejected_by_independent_check":0,"refused_by_a_rule":0,"check_did_not_answer":0,"proposed_without_a_cell":0,"declined_by_site":0}}}