{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/planning-behavior-in-a-recurrent-neural","title":"Planning in a recurrent neural network that plays Sokoban","arxiv_id":"2407.15421","date":"2024-07-22","proceeding":null,"authors":["Mohammad Taufeeque","Philip Quirke","Maximilian Li","Chris Cundy","Aaron David Tucker","Adam Gleave","Adrià Garriga-Alonso"],"abstract":"How a neural network (NN) generalizes to novel situations depends on whether it has learned to select actions heuristically or via a planning process. \"An investigation of model-free planning\" (Guez et al. 2019) found that a recurrent NN (RNN) trained to play Sokoban appears to plan, with extra computation steps improving the RNN's success rate. We replicate and expand on their behavioral analysis, finding the RNN learns to give itself extra computation steps in complex situations by \"pacing\" in cycles. Moreover, we train linear probes that predict the future actions taken by the network and find that intervening on the hidden state using these probes controls the agent's subsequent actions. Leveraging these insights, we perform model surgery, enabling the convolutional NN to generalize beyond its 10x10 architectural limit to arbitrarily sized inputs. The resulting model solves challenging, highly off-distribution levels. We open-source our model and code, and believe the neural network's small size (1.29M parameters) makes it an excellent model organism to deepen our understanding of learned planning.","url_abs":"https://arxiv.org/abs/2407.15421v2","url_pdf":"https://arxiv.org/pdf/2407.15421v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"planning-behavior-in-a-recurrent-neural","repo_url":"https://github.com/alignmentresearch/learned-planner","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":null},{"paper_slug":"planning-behavior-in-a-recurrent-neural","repo_url":"https://github.com/alignmentresearch/train-learned-planner","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"sokoban","task_name":"Sokoban"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.15421","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.15421"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/alignmentresearch/train-learned-planner","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/alignmentresearch/learned-planner","reach":null}],"summary":{"ran_fixture":1,"ran_honours":1,"ran_violates":1,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"c65d2058bbfc8234","entry":"downsample","repo":"alignmentresearch/learned-planner","repo_kind":"official","path":"learned_planner/interp/collect_dataset.py","file_url":"https://github.com/alignmentresearch/learned-planner/blob/HEAD/learned_planner/interp/collect_dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c65d2058bbfc8234"}},{"code_sha256_prefix":"282910eafb91ddc1","entry":"get_probe_preds","repo":"alignmentresearch/learned-planner","repo_kind":"official","path":"plot/behavior_analysis.py","file_url":"https://github.com/alignmentresearch/learned-planner/blob/HEAD/plot/behavior_analysis.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"282910eafb91ddc1"}},{"code_sha256_prefix":"8065cb438e9a52d2","entry":"is_grid_wise_dataset","repo":"alignmentresearch/learned-planner","repo_kind":"official","path":"learned_planner/interp/train_probes.py","file_url":"https://github.com/alignmentresearch/learned-planner/blob/HEAD/learned_planner/interp/train_probes.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8065cb438e9a52d2"}},{"code_sha256_prefix":"dec2d4521ca0eadc","entry":"unreplicate","repo":"alignmentresearch/train-learned-planner","repo_kind":"official","path":"cleanba/cleanba_impala.py","file_url":"https://github.com/alignmentresearch/train-learned-planner/blob/HEAD/cleanba/cleanba_impala.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"dec2d4521ca0eadc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}