{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/seek-in-the-dark-reasoning-via-test-time","title":"Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space","arxiv_id":"2505.13308","date":"2025-05-19","proceeding":null,"authors":["Hengli Li","Chenxi Li","Tong Wu","Xuekai Zhu","Yuxuan Wang","Zhaoxin Yu","Eric Hanchen Jiang","Song-Chun Zhu","Zixia Jia","Ying Nian Wu","Zilong Zheng"],"abstract":"Reasoning ability, a core component of human intelligence, continues to pose a significant challenge for Large Language Models (LLMs) in the pursuit of AGI. Although model performance has improved under the training scaling law, significant challenges remain, particularly with respect to training algorithms, such as catastrophic forgetting, and the limited availability of novel training data. As an alternative, test-time scaling enhances reasoning performance by increasing test-time computation without parameter updating. Unlike prior methods in this paradigm focused on token space, we propose leveraging latent space for more effective reasoning and better adherence to the test-time scaling law. We introduce LatentSeek, a novel framework that enhances LLM reasoning through Test-Time Instance-level Adaptation (TTIA) within the model's latent space. Specifically, LatentSeek leverages policy gradient to iteratively update latent representations, guided by self-generated reward signals. LatentSeek is evaluated on a range of reasoning benchmarks, including GSM8K, MATH-500, and AIME2024, across multiple LLM architectures. Results show that LatentSeek consistently outperforms strong baselines, such as Chain-of-Thought prompting and fine-tuning-based methods. Furthermore, our analysis demonstrates that LatentSeek is highly efficient, typically converging within a few iterations for problems of average complexity, while also benefiting from additional iterations, thereby highlighting the potential of test-time scaling in the latent space. These findings position LatentSeek as a lightweight, scalable, and effective solution for enhancing the reasoning capabilities of LLMs.","url_abs":"https://arxiv.org/abs/2505.13308v1","url_pdf":"https://arxiv.org/pdf/2505.13308v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"seek-in-the-dark-reasoning-via-test-time","repo_url":"https://github.com/bigai-nlco/latentseek","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"gsm8k","task_name":"GSM8K"},{"task_slug":"math","task_name":"Math"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2505.13308","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.13308"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bigai-nlco/latentseek","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_violates":1,"unverified":15},"by_repo_kind":{"official":{"samples":16,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"20063c675d638a77","entry":"is_equiv","repo":"bigai-nlco/latentseek","repo_kind":"official","path":"src/extract_judge_answer/math_equivalent_MATH.py","file_url":"https://github.com/bigai-nlco/latentseek/blob/HEAD/src/extract_judge_answer/math_equivalent_MATH.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"20063c675d638a77"}},{"code_sha256_prefix":"7ced33ff18e50b62","entry":"AIME_2024_prompt","repo":"bigai-nlco/latentseek","repo_kind":"official","path":"src/prompts/solver_prompts.py","file_url":"https://github.com/bigai-nlco/latentseek/blob/HEAD/src/prompts/solver_prompts.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7ced33ff18e50b62"}},{"code_sha256_prefix":"c7427018645757d3","entry":"MATH_500_prompt","repo":"bigai-nlco/latentseek","repo_kind":"official","path":"src/prompts/solver_prompts.py","file_url":"https://github.com/bigai-nlco/latentseek/blob/HEAD/src/prompts/solver_prompts.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c7427018645757d3"}},{"code_sha256_prefix":"809c0178a7ae90ff","entry":"choice_answer_clean","repo":"bigai-nlco/latentseek","repo_kind":"official","path":"src/extract_judge_answer/grader.py","file_url":"https://github.com/bigai-nlco/latentseek/blob/HEAD/src/extract_judge_answer/grader.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"809c0178a7ae90ff"}},{"code_sha256_prefix":"29ac3caee056dbbd","entry":"choice_answer_clean","repo":"bigai-nlco/latentseek","repo_kind":"official","path":"src/extract_judge_answer/parse_utils_qwen.py","file_url":"https://github.com/bigai-nlco/latentseek/blob/HEAD/src/extract_judge_answer/parse_utils_qwen.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"29ac3caee056dbbd"}},{"code_sha256_prefix":"9675ce6b40daf814","entry":"convert_expr","repo":"bigai-nlco/latentseek","repo_kind":"official","path":"src/extract_judge_answer/latex2sympy/latex2sympy2.py","file_url":"https://github.com/bigai-nlco/latentseek/blob/HEAD/src/extract_judge_answer/latex2sympy/latex2sympy2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9675ce6b40daf814"}},{"code_sha256_prefix":"d3b7a1bd2b1b514a","entry":"convert_relation","repo":"bigai-nlco/latentseek","repo_kind":"official","path":"src/extract_judge_answer/latex2sympy/latex2sympy2.py","file_url":"https://github.com/bigai-nlco/latentseek/blob/HEAD/src/extract_judge_answer/latex2sympy/latex2sympy2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d3b7a1bd2b1b514a"}},{"code_sha256_prefix":"6b13e7afc2042384","entry":"extract_answer","repo":"bigai-nlco/latentseek","repo_kind":"official","path":"src/extract_judge_answer/utils.py","file_url":"https://github.com/bigai-nlco/latentseek/blob/HEAD/src/extract_judge_answer/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6b13e7afc2042384"}},{"code_sha256_prefix":"55245bd240cd59a0","entry":"extract_true_answer","repo":"bigai-nlco/latentseek","repo_kind":"official","path":"src/extract_judge_answer/utils.py","file_url":"https://github.com/bigai-nlco/latentseek/blob/HEAD/src/extract_judge_answer/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"55245bd240cd59a0"}},{"code_sha256_prefix":"a5b1d036650981f3","entry":"get_vera_prompt","repo":"bigai-nlco/latentseek","repo_kind":"official","path":"src/prompts/vera_prompts.py","file_url":"https://github.com/bigai-nlco/latentseek/blob/HEAD/src/prompts/vera_prompts.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a5b1d036650981f3"}},{"code_sha256_prefix":"b82d92df5b302601","entry":"gsm8k_prompt","repo":"bigai-nlco/latentseek","repo_kind":"official","path":"src/prompts/solver_prompts.py","file_url":"https://github.com/bigai-nlco/latentseek/blob/HEAD/src/prompts/solver_prompts.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b82d92df5b302601"}},{"code_sha256_prefix":"fb37d3c84b18e368","entry":"is_digit","repo":"bigai-nlco/latentseek","repo_kind":"official","path":"src/extract_judge_answer/grader.py","file_url":"https://github.com/bigai-nlco/latentseek/blob/HEAD/src/extract_judge_answer/grader.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fb37d3c84b18e368"}},{"code_sha256_prefix":"699bf6581fe5aa36","entry":"latex2sympy","repo":"bigai-nlco/latentseek","repo_kind":"official","path":"src/extract_judge_answer/latex2sympy/latex2sympy2.py","file_url":"https://github.com/bigai-nlco/latentseek/blob/HEAD/src/extract_judge_answer/latex2sympy/latex2sympy2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"699bf6581fe5aa36"}},{"code_sha256_prefix":"0dfb6b192649acd6","entry":"optimized_generation","repo":"bigai-nlco/latentseek","repo_kind":"official","path":"src/opt_generation.py","file_url":"https://github.com/bigai-nlco/latentseek/blob/HEAD/src/opt_generation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0dfb6b192649acd6"}},{"code_sha256_prefix":"7334a52286348c49","entry":"original_generation","repo":"bigai-nlco/latentseek","repo_kind":"official","path":"src/ori_generation.py","file_url":"https://github.com/bigai-nlco/latentseek/blob/HEAD/src/ori_generation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7334a52286348c49"}},{"code_sha256_prefix":"7d4b7f8c3875b126","entry":"parse_digits","repo":"bigai-nlco/latentseek","repo_kind":"official","path":"src/extract_judge_answer/grader.py","file_url":"https://github.com/bigai-nlco/latentseek/blob/HEAD/src/extract_judge_answer/grader.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7d4b7f8c3875b126"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}