{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2603-00623","title":"TraceSIR: A Multi-Agent Framework for Structured Analysis and Reporting of Agentic Execution Traces","arxiv_id":"2603.00623","date":"2026-02-28","proceeding":null,"authors":["Shu-Xun Yang","Cunxiang Wang","Haoke Zhang","Wenbo Yu","Lindong Wu","Jiayi Gui","Dayong Yang","Yukuo Cen","Zhuoer Feng","Bosi Wen","Yidong Wang","Lucen Zhong","Jiamin Ren","Linfeng Zhang","Jie Tang"],"abstract":"Agentic systems augment large language models with external tools and iterative decision making, enabling complex tasks such as deep research, function calling, and coding. However, their long and intricate execution traces make failure diagnosis and root cause analysis extremely challenging. Manual inspection does not scale, while directly applying LLMs to raw traces is hindered by input length limits and unreliable reasoning. Focusing solely on final task outcomes further discards critical behavioral information required for accurate issue localization. To address these issues, we propose TraceSIR, a multi-agent framework for structured analysis and reporting of agentic execution traces. TraceSIR coordinates three specialized agents: (1) StructureAgent, which introduces a novel abstraction format, TraceFormat, to compress execution traces while preserving essential behavioral information; (2) InsightAgent, which performs fine-grained diagnosis including issue localization, root cause analysis, and optimization suggestions; (3) ReportAgent, which aggregates insights across task instances and generates comprehensive analysis reports. To evaluate TraceSIR, we construct TraceBench, covering three real-world agentic scenarios, and introduce ReportEval, an evaluation protocol for assessing the quality and usability of analysis reports aligned with industry needs. Experiments show that TraceSIR consistently produces coherent, informative, and actionable reports, significantly outperforming existing approaches across all evaluation dimensions. Our project and video are publicly available at https://github.com/SHU-XUN/TraceSIR.","url_abs":"https://arxiv.org/abs/2603.00623","url_pdf":"https://arxiv.org/pdf/2603.00623","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2603.00623","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2603.00623"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/SHU-XUN/TraceSIR","reach":null}],"summary":{"ran":2,"ran_fixture":1,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"535bc1af3796913b","entry":"LLMAgentAPI","repo":"SHU-XUN/TraceSIR","repo_kind":"found_in_text","path":"pipeline.py","file_url":"https://github.com/SHU-XUN/TraceSIR/blob/HEAD/pipeline.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"535bc1af3796913b"}},{"code_sha256_prefix":"8a1492ab96ed6b30","entry":"LLMAgentToolAPI","repo":"SHU-XUN/TraceSIR","repo_kind":"found_in_text","path":"pipeline.py","file_url":"https://github.com/SHU-XUN/TraceSIR/blob/HEAD/pipeline.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8a1492ab96ed6b30"}},{"code_sha256_prefix":"509d3b2efc554c1d","entry":"make_logger","repo":"SHU-XUN/TraceSIR","repo_kind":"found_in_text","path":"pipeline.py","file_url":"https://github.com/SHU-XUN/TraceSIR/blob/HEAD/pipeline.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"509d3b2efc554c1d"}},{"code_sha256_prefix":"8208bbf74f3b0d60","entry":"process_trace_folder","repo":"SHU-XUN/TraceSIR","repo_kind":"found_in_text","path":"pipeline.py","file_url":"https://github.com/SHU-XUN/TraceSIR/blob/HEAD/pipeline.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8208bbf74f3b0d60"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}