{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multihiertt-numerical-reasoning-over-multi-1","title":"MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Data","arxiv_id":"2206.01347","date":"2022-06-03","proceeding":"ACL 2022 5","authors":["Yilun Zhao","Yunxiang Li","Chenying Li","Rui Zhang"],"abstract":"Numerical reasoning over hybrid data containing both textual and tabular content (e.g., financial reports) has recently attracted much attention in the NLP community. However, existing question answering (QA) benchmarks over hybrid data only include a single flat table in each document and thus lack examples of multi-step numerical reasoning across multiple hierarchical tables. To facilitate data analytical progress, we construct a new large-scale benchmark, MultiHiertt, with QA pairs over Multi Hierarchical Tabular and Textual data. MultiHiertt is built from a wealth of financial reports and has the following unique characteristics: 1) each document contain multiple tables and longer unstructured texts; 2) most of tables contained are hierarchical; 3) the reasoning process required for each question is more complex and challenging than existing benchmarks; and 4) fine-grained annotations of reasoning processes and supporting facts are provided to reveal complex numerical reasoning. We further introduce a novel QA model termed MT2Net, which first applies facts retrieving to extract relevant supporting facts from both tables and text and then uses a reasoning module to perform symbolic reasoning over retrieved facts. We conduct comprehensive experiments on various baselines. The experimental results show that MultiHiertt presents a strong challenge for existing baselines whose results lag far behind the performance of human experts. The dataset and code are publicly available at https://github.com/psunlpgroup/MultiHiertt.","url_abs":"https://arxiv.org/abs/2206.01347v1","url_pdf":"https://arxiv.org/pdf/2206.01347v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multihiertt-numerical-reasoning-over-multi-1","repo_url":"https://github.com/psunlpgroup/multihiertt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2206.01347","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.01347"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/psunlpgroup/multihiertt","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":5},"by_repo_kind":{"official":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bcd32c30c7e7c795","entry":"combine_predictions","repo":"psunlpgroup/multihiertt","repo_kind":"official","path":"evaluate.py","file_url":"https://github.com/psunlpgroup/multihiertt/blob/HEAD/evaluate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bcd32c30c7e7c795"}},{"code_sha256_prefix":"49517bcb4e57f9c9","entry":"evaluate_program_result","repo":"psunlpgroup/multihiertt","repo_kind":"official","path":"evaluate.py","file_url":"https://github.com/psunlpgroup/multihiertt/blob/HEAD/evaluate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"49517bcb4e57f9c9"}},{"code_sha256_prefix":"191010c8de1ad5c5","entry":"evaluate_span_program_result","repo":"psunlpgroup/multihiertt","repo_kind":"official","path":"evaluate.py","file_url":"https://github.com/psunlpgroup/multihiertt/blob/HEAD/evaluate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"191010c8de1ad5c5"}},{"code_sha256_prefix":"65c5b5b5c7482e97","entry":"get_context","repo":"psunlpgroup/multihiertt","repo_kind":"official","path":"convert_retriever_result.py","file_url":"https://github.com/psunlpgroup/multihiertt/blob/HEAD/convert_retriever_result.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"65c5b5b5c7482e97"}},{"code_sha256_prefix":"4403574d9e6aacf8","entry":"get_table_sent_map","repo":"psunlpgroup/multihiertt","repo_kind":"official","path":"convert_retriever_result.py","file_url":"https://github.com/psunlpgroup/multihiertt/blob/HEAD/convert_retriever_result.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4403574d9e6aacf8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}