{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/can-language-models-learn-analogical","title":"Can language models learn analogical reasoning? Investigating training objectives and comparisons to human performance","arxiv_id":"2310.05597","date":"2023-10-09","proceeding":null,"authors":["Molly R. Petersen","Lonneke van der Plas"],"abstract":"While analogies are a common way to evaluate word embeddings in NLP, it is also of interest to investigate whether or not analogical reasoning is a task in itself that can be learned. In this paper, we test several ways to learn basic analogical reasoning, specifically focusing on analogies that are more typical of what is used to evaluate analogical reasoning in humans than those in commonly used NLP benchmarks. Our experiments find that models are able to learn analogical reasoning, even with a small amount of data. We additionally compare our models to a dataset with a human baseline, and find that after training, models approach human performance.","url_abs":"https://arxiv.org/abs/2310.05597v4","url_pdf":"https://arxiv.org/pdf/2310.05597v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"can-language-models-learn-analogical","repo_url":"https://github.com/idiap/analogy_learning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"word-embeddings","task_name":"Word Embeddings"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.05597","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.05597"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/kudkudak/word-embeddings-benchmarks","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4,"unverified":3},"by_repo_kind":{"found_in_text":{"samples":7,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e1577d6a21ef14d6","entry":"any2utf8","repo":"kudkudak/word-embeddings-benchmarks","repo_kind":"found_in_text","path":"web/utils.py","file_url":"https://github.com/kudkudak/word-embeddings-benchmarks/blob/HEAD/web/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e1577d6a21ef14d6"}},{"code_sha256_prefix":"70134ccbd3ed1575","entry":"calculate_purity","repo":"kudkudak/word-embeddings-benchmarks","repo_kind":"found_in_text","path":"web/evaluate.py","file_url":"https://github.com/kudkudak/word-embeddings-benchmarks/blob/HEAD/web/evaluate.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"70134ccbd3ed1575"}},{"code_sha256_prefix":"dfdbeb9cc00ba5f0","entry":"count","repo":"kudkudak/word-embeddings-benchmarks","repo_kind":"found_in_text","path":"web/vocabulary.py","file_url":"https://github.com/kudkudak/word-embeddings-benchmarks/blob/HEAD/web/vocabulary.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dfdbeb9cc00ba5f0"}},{"code_sha256_prefix":"df0ff2e74440c1f9","entry":"readlinkabs","repo":"kudkudak/word-embeddings-benchmarks","repo_kind":"found_in_text","path":"web/datasets/utils.py","file_url":"https://github.com/kudkudak/word-embeddings-benchmarks/blob/HEAD/web/datasets/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df0ff2e74440c1f9"}},{"code_sha256_prefix":"5916ccc43e1a1c72","entry":"fetch_GloVe","repo":"kudkudak/word-embeddings-benchmarks","repo_kind":"found_in_text","path":"web/embeddings.py","file_url":"https://github.com/kudkudak/word-embeddings-benchmarks/blob/HEAD/web/embeddings.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5916ccc43e1a1c72"}},{"code_sha256_prefix":"3b58ab51907af864","entry":"fetch_HPCA","repo":"kudkudak/word-embeddings-benchmarks","repo_kind":"found_in_text","path":"web/embeddings.py","file_url":"https://github.com/kudkudak/word-embeddings-benchmarks/blob/HEAD/web/embeddings.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b58ab51907af864"}},{"code_sha256_prefix":"df1baac974fb5db7","entry":"load_embedding","repo":"kudkudak/word-embeddings-benchmarks","repo_kind":"found_in_text","path":"web/embeddings.py","file_url":"https://github.com/kudkudak/word-embeddings-benchmarks/blob/HEAD/web/embeddings.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df1baac974fb5db7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}