{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hopping-too-late-exploring-the-limitations-of","title":"Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queries","arxiv_id":"2406.12775","date":"2024-06-18","proceeding":null,"authors":["Eden Biran","Daniela Gottesman","Sohee Yang","Mor Geva","Amir Globerson"],"abstract":"Large language models (LLMs) can solve complex multi-step problems, but little is known about how these computations are implemented internally. Motivated by this, we study how LLMs answer multi-hop queries such as \"The spouse of the performer of Imagine is\". These queries require two information extraction steps: a latent one for resolving the first hop (\"the performer of Imagine\") into the bridge entity (John Lennon), and another for resolving the second hop (\"the spouse of John Lennon\") into the target entity (Yoko Ono). Understanding how the latent step is computed internally is key to understanding the overall computation. By carefully analyzing the internal computations of transformer-based LLMs, we discover that the bridge entity is resolved in the early layers of the model. Then, only after this resolution, the two-hop query is solved in the later layers. Because the second hop commences in later layers, there could be cases where these layers no longer encode the necessary knowledge for correctly predicting the answer. Motivated by this, we propose a novel \"back-patching\" analysis method whereby a hidden representation from a later layer is patched back to an earlier layer. We find that in up to 66% of previously incorrect cases there exists a back-patch that results in the correct generation of the answer, showing that the later layers indeed sometimes lack the needed functionality. Overall, our methods and findings open further opportunities for understanding and improving latent reasoning in transformer-based LLMs.","url_abs":"https://arxiv.org/abs/2406.12775v2","url_pdf":"https://arxiv.org/pdf/2406.12775v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hopping-too-late-exploring-the-limitations-of","repo_url":"https://github.com/edenbiran/HoppingTooLate","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"favor","method_name":"FAVOR+"},{"method_slug":"performer","method_name":"Performer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.12775","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.12775"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/edenbiran/HoppingTooLate","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":10,"unverified":2},"by_repo_kind":{"official":{"samples":12,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0bad694c005e385c","entry":"create_target_prompt","repo":"edenbiran/HoppingTooLate","repo_kind":"official","path":"src/patch_activations.py","file_url":"https://github.com/edenbiran/HoppingTooLate/blob/HEAD/src/patch_activations.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0bad694c005e385c"}},{"code_sha256_prefix":"3820700493cb2dd1","entry":"find_layers_by_classification","repo":"edenbiran/HoppingTooLate","repo_kind":"official","path":"src/classify_results.py","file_url":"https://github.com/edenbiran/HoppingTooLate/blob/HEAD/src/classify_results.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3820700493cb2dd1"}},{"code_sha256_prefix":"e3649917ac1cd8d7","entry":"get_entities_per_type","repo":"edenbiran/HoppingTooLate","repo_kind":"official","path":"src/create_dataset.py","file_url":"https://github.com/edenbiran/HoppingTooLate/blob/HEAD/src/create_dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e3649917ac1cd8d7"}},{"code_sha256_prefix":"c932a81e28f8b5ec","entry":"get_entity_counts","repo":"edenbiran/HoppingTooLate","repo_kind":"official","path":"src/analyze_results.py","file_url":"https://github.com/edenbiran/HoppingTooLate/blob/HEAD/src/analyze_results.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c932a81e28f8b5ec"}},{"code_sha256_prefix":"cdc175a60f83d924","entry":"get_layer_names","repo":"edenbiran/HoppingTooLate","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/edenbiran/HoppingTooLate/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cdc175a60f83d924"}},{"code_sha256_prefix":"97485dbb4fcb5def","entry":"get_one_hop","repo":"edenbiran/HoppingTooLate","repo_kind":"official","path":"src/create_dataset.py","file_url":"https://github.com/edenbiran/HoppingTooLate/blob/HEAD/src/create_dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"97485dbb4fcb5def"}},{"code_sha256_prefix":"4fd83ad38d7e72b8","entry":"get_popular_entities_per_type","repo":"edenbiran/HoppingTooLate","repo_kind":"official","path":"src/create_dataset.py","file_url":"https://github.com/edenbiran/HoppingTooLate/blob/HEAD/src/create_dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4fd83ad38d7e72b8"}},{"code_sha256_prefix":"e12cea8b6661a026","entry":"get_stage_layers","repo":"edenbiran/HoppingTooLate","repo_kind":"official","path":"src/analyze_results.py","file_url":"https://github.com/edenbiran/HoppingTooLate/blob/HEAD/src/analyze_results.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e12cea8b6661a026"}},{"code_sha256_prefix":"9ca91681e0ed00b8","entry":"get_sublayer_counts","repo":"edenbiran/HoppingTooLate","repo_kind":"official","path":"src/analyze_results.py","file_url":"https://github.com/edenbiran/HoppingTooLate/blob/HEAD/src/analyze_results.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9ca91681e0ed00b8"}},{"code_sha256_prefix":"8164e26307c3a60b","entry":"load_tokenizer","repo":"edenbiran/HoppingTooLate","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/edenbiran/HoppingTooLate/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8164e26307c3a60b"}},{"code_sha256_prefix":"5c1337c4e1ef4149","entry":"find_tokens","repo":"edenbiran/HoppingTooLate","repo_kind":"official","path":"src/patched_generation.py","file_url":"https://github.com/edenbiran/HoppingTooLate/blob/HEAD/src/patched_generation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5c1337c4e1ef4149"}},{"code_sha256_prefix":"9d02aac80c0eae1a","entry":"load_model","repo":"edenbiran/HoppingTooLate","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/edenbiran/HoppingTooLate/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9d02aac80c0eae1a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}