{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/two-is-better-than-many-binary-classification","title":"Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering","arxiv_id":"2210.16495","date":"2022-10-29","proceeding":null,"authors":["Deepanway Ghosal","Navonil Majumder","Rada Mihalcea","Soujanya Poria"],"abstract":"We propose a simple refactoring of multi-choice question answering (MCQA) tasks as a series of binary classifications. The MCQA task is generally performed by scoring each (question, answer) pair normalized over all the pairs, and then selecting the answer from the pair that yield the highest score. For n answer choices, this is equivalent to an n-class classification setup where only one class (true answer) is correct. We instead show that classifying (question, true answer) as positive instances and (question, false answer) as negative instances is significantly more effective across various models and datasets. We show the efficacy of our proposed approach in different tasks -- abductive reasoning, commonsense question answering, science question answering, and sentence completion. Our DeBERTa binary classification model reaches the top or close to the top performance on public leaderboards for these tasks. The source code of the proposed approach is available at https://github.com/declare-lab/TEAM.","url_abs":"https://arxiv.org/abs/2210.16495v1","url_pdf":"https://arxiv.org/pdf/2210.16495v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"two-is-better-than-many-binary-classification","repo_url":"https://github.com/declare-lab/team","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"binary-classification","task_name":"Binary Classification"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"science-question-answering","task_name":"Science Question Answering"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"sentence-completion","task_name":"Sentence Completion"}],"methods":[{"method_slug":"deberta","method_name":"DeBERTa"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/question-answering-on-piqa","task":"Question Answering","dataset":"PIQA","model":"DeBERTa-Large 304M","rank_in_archive_order":5,"of":67,"metrics":{"Accuracy":"87.4"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-piqa","task":"Question Answering","dataset":"PIQA","model":"DeBERTa-Large 304M (classification-based)","rank_in_archive_order":9,"of":67,"metrics":{"Accuracy":"85.9"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-social-iqa","task":"Question Answering","dataset":"SIQA","model":"DeBERTa-Large 304M","rank_in_archive_order":6,"of":24,"metrics":{"Accuracy":"80.2"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-social-iqa","task":"Question Answering","dataset":"SIQA","model":"DeBERTa-Large 304M (classification-based)","rank_in_archive_order":7,"of":24,"metrics":{"Accuracy":"79.9"},"uses_additional_data":false},{"leaderboard":"/sota/sentence-completion-on-hellaswag","task":"Sentence Completion","dataset":"HellaSwag","model":"DeBERTa-Large 304M (classification-based)","rank_in_archive_order":3,"of":89,"metrics":{"Accuracy":"95.6"},"uses_additional_data":false},{"leaderboard":"/sota/sentence-completion-on-hellaswag","task":"Sentence Completion","dataset":"HellaSwag","model":"DeBERTa-Large 304M","rank_in_archive_order":6,"of":89,"metrics":{"Accuracy":"94.7"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2210.16495","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.16495"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/declare-lab/TEAM","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":6},"by_repo_kind":{"official":{"samples":7,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4dab6811065fbe1c","entry":"configure_optimizer","repo":"declare-lab/TEAM","repo_kind":"official","path":"train_cicero.py","file_url":"https://github.com/declare-lab/TEAM/blob/HEAD/train_cicero.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4dab6811065fbe1c"}},{"code_sha256_prefix":"25695845c448eef9","entry":"configure_dataloaders","repo":"declare-lab/TEAM","repo_kind":"official","path":"train_cicero.py","file_url":"https://github.com/declare-lab/TEAM/blob/HEAD/train_cicero.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"25695845c448eef9"}},{"code_sha256_prefix":"eedd86173a2d52d7","entry":"configure_dataloaders","repo":"declare-lab/TEAM","repo_kind":"official","path":"train_cicero2.py","file_url":"https://github.com/declare-lab/TEAM/blob/HEAD/train_cicero2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eedd86173a2d52d7"}},{"code_sha256_prefix":"5d544b01234bdcee","entry":"configure_dataloaders","repo":"declare-lab/TEAM","repo_kind":"official","path":"train_cosmosqa.py","file_url":"https://github.com/declare-lab/TEAM/blob/HEAD/train_cosmosqa.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5d544b01234bdcee"}},{"code_sha256_prefix":"6b021e1d8b19988c","entry":"configure_dataloaders","repo":"declare-lab/TEAM","repo_kind":"official","path":"train_csqa.py","file_url":"https://github.com/declare-lab/TEAM/blob/HEAD/train_csqa.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6b021e1d8b19988c"}},{"code_sha256_prefix":"ffc49c4218a2e155","entry":"configure_dataloaders","repo":"declare-lab/TEAM","repo_kind":"official","path":"train_csqa2.py","file_url":"https://github.com/declare-lab/TEAM/blob/HEAD/train_csqa2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ffc49c4218a2e155"}},{"code_sha256_prefix":"83ef42f1f5c80dc5","entry":"configure_scheduler","repo":"declare-lab/TEAM","repo_kind":"official","path":"train_cicero.py","file_url":"https://github.com/declare-lab/TEAM/blob/HEAD/train_cicero.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"83ef42f1f5c80dc5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}