{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/convfinqa-exploring-the-chain-of-numerical","title":"ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering","arxiv_id":"2210.03849","date":"2022-10-07","proceeding":null,"authors":["Zhiyu Chen","Shiyang Li","Charese Smiley","Zhiqiang Ma","Sameena Shah","William Yang Wang"],"abstract":"With the recent advance in large pre-trained language models, researchers have achieved record performances in NLP tasks that mostly focus on language pattern matching. The community is experiencing the shift of the challenge from how to model language to the imitation of complex reasoning abilities like human beings. In this work, we investigate the application domain of finance that involves real-world, complex numerical reasoning. We propose a new large-scale dataset, ConvFinQA, aiming to study the chain of numerical reasoning in conversational question answering. Our dataset poses great challenge in modeling long-range, complex numerical reasoning paths in real-world conversations. We conduct comprehensive experiments and analyses with both the neural symbolic methods and the prompting-based methods, to provide insights into the reasoning mechanisms of these two divisions. We believe our new dataset should serve as a valuable resource to push forward the exploration of real-world, complex reasoning tasks as the next research focus. Our dataset and code is publicly available at https://github.com/czyssrs/ConvFinQA.","url_abs":"https://arxiv.org/abs/2210.03849v1","url_pdf":"https://arxiv.org/pdf/2210.03849v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"convfinqa-exploring-the-chain-of-numerical","repo_url":"https://github.com/czyssrs/convfinqa","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"conversational-question-answering","task_name":"Conversational Question Answering"},{"task_slug":"question-answering","task_name":"Question Answering"}],"methods":[],"datasets_introduced":[{"slug":"convfinqa","name":"ConvFinQA","full_name":"Conversational Finance Question Answering"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/conversational-question-answering-on","task":"Conversational Question Answering","dataset":"ConvFinQA","model":"FinQANet (RoBERTa-large)","rank_in_archive_order":2,"of":2,"metrics":{"Execution Accuracy":"68.90","Program Accuracy":"68.24"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-convfinqa","task":"Question Answering","dataset":"ConvFinQA","model":"FinQANet (RoBERTa-large)","rank_in_archive_order":2,"of":3,"metrics":{"Execution Accuracy":"68.9"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2210.03849","atlas_url":"https://app.syntology.ai/?focus=2210.03849","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.03849"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/czyssrs/convfinqa","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"ran":5,"unverified":2},"by_repo_kind":{"official":{"samples":8,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8d7ec010c29e813b","entry":"format_time","repo":"czyssrs/convfinqa","repo_kind":"official","path":"code/finqanet_generator/utils.py","file_url":"https://github.com/czyssrs/convfinqa/blob/HEAD/code/finqanet_generator/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8d7ec010c29e813b"}},{"code_sha256_prefix":"21ff8e9f6a35119e","entry":"prog_token_to_indices","repo":"czyssrs/convfinqa","repo_kind":"official","path":"code/finqanet_generator/finqa_utils.py","file_url":"https://github.com/czyssrs/convfinqa/blob/HEAD/code/finqanet_generator/finqa_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"21ff8e9f6a35119e"}},{"code_sha256_prefix":"5312b6b693f09d8e","entry":"read_txt","repo":"czyssrs/convfinqa","repo_kind":"official","path":"code/finqanet_generator/utils.py","file_url":"https://github.com/czyssrs/convfinqa/blob/HEAD/code/finqanet_generator/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5312b6b693f09d8e"}},{"code_sha256_prefix":"edc24c5af3e1a7cd","entry":"remove_space","repo":"czyssrs/convfinqa","repo_kind":"official","path":"code/utils/general_utils.py","file_url":"https://github.com/czyssrs/convfinqa/blob/HEAD/code/utils/general_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"edc24c5af3e1a7cd"}},{"code_sha256_prefix":"227ac417efac55e8","entry":"str_to_num","repo":"czyssrs/convfinqa","repo_kind":"official","path":"code/finqanet_generator/finqa_utils.py","file_url":"https://github.com/czyssrs/convfinqa/blob/HEAD/code/finqanet_generator/finqa_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"227ac417efac55e8"}},{"code_sha256_prefix":"3c93e9aa942bd60c","entry":"table_row_to_text","repo":"czyssrs/convfinqa","repo_kind":"official","path":"code/utils/general_utils.py","file_url":"https://github.com/czyssrs/convfinqa/blob/HEAD/code/utils/general_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3c93e9aa942bd60c"}},{"code_sha256_prefix":"c7368d407afb9277","entry":"cleanhtml","repo":"czyssrs/convfinqa","repo_kind":"official","path":"code/utils/general_utils.py","file_url":"https://github.com/czyssrs/convfinqa/blob/HEAD/code/utils/general_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c7368d407afb9277"}},{"code_sha256_prefix":"6b93716e4bdcdfde","entry":"indices_to_prog","repo":"czyssrs/convfinqa","repo_kind":"official","path":"code/finqanet_generator/finqa_utils.py","file_url":"https://github.com/czyssrs/convfinqa/blob/HEAD/code/finqanet_generator/finqa_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6b93716e4bdcdfde"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}