{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-simple-method-for-commonsense-reasoning","title":"A Simple Method for Commonsense Reasoning","arxiv_id":"1806.02847","date":"2018-06-07","proceeding":null,"authors":["Trieu H. Trinh","Quoc V. Le"],"abstract":"Commonsense reasoning is a long-standing challenge for deep learning. For example, it is difficult to use neural networks to tackle the Winograd Schema dataset (Levesque et al., 2011). In this paper, we present a simple method for commonsense reasoning with neural networks, using unsupervised learning. Key to our method is the use of language models, trained on a massive amount of unlabled data, to score multiple choice questions posed by commonsense reasoning tests. On both Pronoun Disambiguation and Winograd Schema challenges, our models outperform previous state-of-the-art methods by a large margin, without using expensive annotated knowledge bases or hand-engineered features. We train an array of large RNN language models that operate at word or character level on LM-1-Billion, CommonCrawl, SQuAD, Gutenberg Books, and a customized corpus for this task and show that diversity of training data plays an important role in test performance. Further analysis also shows that our system successfully discovers important features of the context that decide the correct answer, indicating a good grasp of commonsense knowledge.","url_abs":"https://arxiv.org/abs/1806.02847v2","url_pdf":"https://arxiv.org/pdf/1806.02847v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-simple-method-for-commonsense-reasoning","repo_url":"https://github.com/gabimelo/portuguese_wsc","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"a-simple-method-for-commonsense-reasoning","repo_url":"https://github.com/tensorflow/models/tree/master/research/lm_commonsense","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"common-sense-reasoning","task_name":"Common Sense Reasoning"},{"task_slug":"coreference-resolution","task_name":"Coreference Resolution"},{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"multiple-choice","task_name":"Multiple-choice"},{"task_slug":"natural-language-understanding","task_name":"Natural Language Understanding"}],"methods":[],"datasets_introduced":[{"slug":"cc-stories","name":"CC-Stories","full_name":"CC-Stories"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/coreference-resolution-on-winograd-schema","task":"Coreference Resolution","dataset":"Winograd Schema Challenge","model":"Ensemble of 14 LMs","rank_in_archive_order":45,"of":82,"metrics":{"Accuracy":"63.7"},"uses_additional_data":false},{"leaderboard":"/sota/coreference-resolution-on-winograd-schema","task":"Coreference Resolution","dataset":"Winograd Schema Challenge","model":"Word-level CNN+LSTM (partial scoring)","rank_in_archive_order":49,"of":82,"metrics":{"Accuracy":"62.6"},"uses_additional_data":false},{"leaderboard":"/sota/coreference-resolution-on-winograd-schema","task":"Coreference Resolution","dataset":"Winograd Schema Challenge","model":"Char-level CNN+LSTM (partial scoring)","rank_in_archive_order":64,"of":82,"metrics":{"Accuracy":"57.9"},"uses_additional_data":false},{"leaderboard":"/sota/natural-language-understanding-on-pdp60","task":"Natural Language Understanding","dataset":"PDP60","model":"Word-level CNN+LSTM (full scoring)","rank_in_archive_order":9,"of":13,"metrics":{"Accuracy":"60.0"},"uses_additional_data":false},{"leaderboard":"/sota/natural-language-understanding-on-pdp60","task":"Natural Language Understanding","dataset":"PDP60","model":"Word-level CNN+LSTM (partial scoring)","rank_in_archive_order":12,"of":13,"metrics":{"Accuracy":"53.3"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1806.02847","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1806.02847"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tensorflow/models/tree/master/research/lm_commonsense","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gabimelo/portuguese_wsc","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":7},"by_repo_kind":{"listed":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0c5ba80cd088ef0c","entry":"clean_tags_for_schema_and_snippet","repo":"gabimelo/portuguese_wsc","repo_kind":"listed","path":"src/winograd_collection_manipulation/wsc_html_parser.py","file_url":"https://github.com/gabimelo/portuguese_wsc/blob/HEAD/src/winograd_collection_manipulation/wsc_html_parser.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0c5ba80cd088ef0c"}},{"code_sha256_prefix":"9b13963c3b04cf46","entry":"filter_good_and_featured_articles","repo":"gabimelo/portuguese_wsc","repo_kind":"listed","path":"src/datasets_manipulation/wikidump.py","file_url":"https://github.com/gabimelo/portuguese_wsc/blob/HEAD/src/datasets_manipulation/wikidump.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9b13963c3b04cf46"}},{"code_sha256_prefix":"bac5119cd0c08a1d","entry":"generate_full_sentences","repo":"gabimelo/portuguese_wsc","repo_kind":"listed","path":"src/winograd_collection_manipulation/wsc_subsets_generation.py","file_url":"https://github.com/gabimelo/portuguese_wsc/blob/HEAD/src/winograd_collection_manipulation/wsc_subsets_generation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bac5119cd0c08a1d"}},{"code_sha256_prefix":"13efa26bab1f01dc","entry":"get_schema_and_snippet_texts","repo":"gabimelo/portuguese_wsc","repo_kind":"listed","path":"src/winograd_collection_manipulation/wsc_html_parser.py","file_url":"https://github.com/gabimelo/portuguese_wsc/blob/HEAD/src/winograd_collection_manipulation/wsc_html_parser.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"13efa26bab1f01dc"}},{"code_sha256_prefix":"096ed1b2d8bad316","entry":"handle_extra_params","repo":"gabimelo/portuguese_wsc","repo_kind":"listed","path":"src/helpers/logger.py","file_url":"https://github.com/gabimelo/portuguese_wsc/blob/HEAD/src/helpers/logger.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"096ed1b2d8bad316"}},{"code_sha256_prefix":"43c3e449fb05495d","entry":"join_content","repo":"gabimelo/portuguese_wsc","repo_kind":"listed","path":"src/winograd_collection_manipulation/wsc_html_parser.py","file_url":"https://github.com/gabimelo/portuguese_wsc/blob/HEAD/src/winograd_collection_manipulation/wsc_html_parser.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"43c3e449fb05495d"}},{"code_sha256_prefix":"571ae1b8d8a82808","entry":"tokenize","repo":"gabimelo/portuguese_wsc","repo_kind":"listed","path":"src/datasets_manipulation/wikidump.py","file_url":"https://github.com/gabimelo/portuguese_wsc/blob/HEAD/src/datasets_manipulation/wikidump.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"571ae1b8d8a82808"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}