{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2608-04374","title":"FinReportBench: Measuring and Improving Institution-Grade Financial Report Generation","arxiv_id":"2608.04374","date":"2026-08-05","proceeding":null,"authors":["Yinghao Tang","Tan Zhenwei","Yiyao Wang","Wanli Gu","Xiaolu Zhang","Jun Zhou","Wei Chen"],"abstract":"Large language models can produce fluent financial analysis, but fluency alone does not establish whether a report is suitable for institutional delivery. We introduce FinReportBench, an expert-grounded benchmark for measuring and improving institution-grade financial report generation. Expert review reveals recurring gaps in report identity, institutional components, source discipline, and visual delivery. We derive a 35-item rubric through expert partial orders, multimodal evidence, and audits of decision boundaries, covering deliverability, report identity, and institutional completeness. Starting from 10,000 balanced Chinese and English financial-research source records, we curate 244 bilingual tasks across three research objects and two input tiers. Each task separates the public query, reconstructed research trajectory, and hidden source packet. Three independent judge families reproduce the expert partial order at near-ceiling rates, showing that bounded, observable criteria support reliable evaluation. Across nine model families, basic deliverability is nearly saturated, while report identity and institutional completeness remain the primary bottlenecks. The largest cross-model gaps concern generation-trace control, information density, and data discipline rather than basic report framing. We then use benchmark-guided skill distillation to turn recurrent failures into reusable generation and self-review constraints. Across five model families, the evolved skill improves mean G1 by 33.85 points and mean G2 by 13.83 points over paired no-skill runs while preserving G0 for every pair. Code and benchmark artifacts are available at https://github.com/MisterBrookT/finreportbench.","url_abs":"https://arxiv.org/abs/2608.04374","url_pdf":"https://arxiv.org/pdf/2608.04374","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2608.04374","atlas_url":null,"mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2608.04374"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/MisterBrookT/finreportbench","reach":null}],"summary":{"ran":1,"ran_draft_wrong":20,"ran_violates":2,"ran_honours":1,"unverified":10},"by_repo_kind":{"found_in_text":{"samples":34,"ran":24,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d6869a2f8afa65af","entry":"_llm_key","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d6869a2f8afa65af"}},{"code_sha256_prefix":"194c9f93a3845373","entry":"build_query_variants","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"194c9f93a3845373"}},{"code_sha256_prefix":"83443b5d91c530c2","entry":"derive_focus","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"83443b5d91c530c2"}},{"code_sha256_prefix":"06dee9524fe6217f","entry":"detect_period","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"06dee9524fe6217f"}},{"code_sha256_prefix":"e5d8393c823a8e25","entry":"evidence_strength","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e5d8393c823a8e25"}},{"code_sha256_prefix":"6ef3a07faa58157f","entry":"extract_report_structure","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6ef3a07faa58157f"}},{"code_sha256_prefix":"54dd79ceb38c9639","entry":"extract_visual_intents","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"54dd79ceb38c9639"}},{"code_sha256_prefix":"460f3f21c6166edb","entry":"find_metric_number","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"460f3f21c6166edb"}},{"code_sha256_prefix":"89556f7232fbb5fc","entry":"infer_chart_family","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"89556f7232fbb5fc"}},{"code_sha256_prefix":"19723a74e741e9b9","entry":"infer_investment_stance","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"19723a74e741e9b9"}},{"code_sha256_prefix":"bb27f23b0801ebeb","entry":"infer_research_problem_domain","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bb27f23b0801ebeb"}},{"code_sha256_prefix":"48b8c14a0690d77f","entry":"infer_sector_evidence_archetype","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"48b8c14a0690d77f"}},{"code_sha256_prefix":"b73083c4afe754b9","entry":"infer_visual_purpose","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b73083c4afe754b9"}},{"code_sha256_prefix":"1cae1b6ab9c50321","entry":"metric_unit_is_usable","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1cae1b6ab9c50321"}},{"code_sha256_prefix":"4689f9f443fa27fc","entry":"metric_unit_is_usable_en","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4689f9f443fa27fc"}},{"code_sha256_prefix":"a29e097f73ef2f0e","entry":"must_include","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a29e097f73ef2f0e"}},{"code_sha256_prefix":"df92a989adbb3239","entry":"natural_query_leak_findings","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df92a989adbb3239"}},{"code_sha256_prefix":"d8984d9524fa3232","entry":"normalize_metric_unit","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d8984d9524fa3232"}},{"code_sha256_prefix":"abeeb022c90d72ee","entry":"normalize_space","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"abeeb022c90d72ee"}},{"code_sha256_prefix":"1f812169a90254fc","entry":"normalize_unit_en","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1f812169a90254fc"}},{"code_sha256_prefix":"052bdc99eb62b3f8","entry":"task_id_for","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"052bdc99eb62b3f8"}},{"code_sha256_prefix":"6890fde4b0e19b99","entry":"title_tokens","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6890fde4b0e19b99"}},{"code_sha256_prefix":"2f57c8e798a2d1e6","entry":"to_number","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2f57c8e798a2d1e6"}},{"code_sha256_prefix":"ef7fa1ce3390e81f","entry":"truncate","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ef7fa1ce3390e81f"}},{"code_sha256_prefix":"8e4d4e0cce24d768","entry":"_facts_to_mainlines_en","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8e4d4e0cce24d768"}},{"code_sha256_prefix":"e8653a43c6606470","entry":"_llm_ci_extract","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e8653a43c6606470"}},{"code_sha256_prefix":"48571947fb109962","entry":"_llm_macro_extract","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"48571947fb109962"}},{"code_sha256_prefix":"37c7b717ba78aa90","entry":"build_case","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"37c7b717ba78aa90"}},{"code_sha256_prefix":"cc2f627ee58b0c3a","entry":"build_sidecar","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cc2f627ee58b0c3a"}},{"code_sha256_prefix":"6afebe9df9823731","entry":"build_taxonomy","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6afebe9df9823731"}},{"code_sha256_prefix":"7bfd1bb3acd28f29","entry":"extract_metric_facts","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7bfd1bb3acd28f29"}},{"code_sha256_prefix":"bad6dbb08427008c","entry":"extract_pdf_pages","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bad6dbb08427008c"}},{"code_sha256_prefix":"d3185fc7de19f7ea","entry":"load_sidecar_or_build","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d3185fc7de19f7ea"}},{"code_sha256_prefix":"dad45d858d6a36f1","entry":"mainlines_from_sidecar","repo":"MisterBrookT/finreportbench","repo_kind":"found_in_text","path":"harness/prepare/reverse_query.py","file_url":"https://github.com/MisterBrookT/finreportbench/blob/HEAD/harness/prepare/reverse_query.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dad45d858d6a36f1"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":{"kind":"leaderboard_placements","source":"Syntology's leaderboard-shaped extractor over the paper's own arXiv-HTML tables: a model pointed at a cell, the number was read from that cell and checked against the board's metric, dataset, split and scale, and an independent check accepted the entry; not reviewed by the paper's authors or the archive's editors","extractor_model":"global.anthropic.claude-sonnet-4-5-20250929-v1:0","verifier_model":null,"prompt_sha":"fa63d4bb9d755694","coverage":{"sentence":"Syntology has checked 6,885 of the 9,623 papers on this site that are newer than the archive; results from the others appear after they are checked.","papers_newer_than_archive":9623,"papers_checked":6885},"entries":[],"not_placed":{"boards":0,"rejected_by_independent_check":0,"refused_by_a_rule":0,"check_did_not_answer":0,"proposed_without_a_cell":0,"declined_by_site":0}}}