{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficient-reasoning-with-hidden-thinking","title":"Efficient Reasoning with Hidden Thinking","arxiv_id":"2501.19201","date":"2025-01-31","proceeding":null,"authors":["Xuan Shen","Yizhou Wang","Xiangxi Shi","Yanzhi Wang","Pu Zhao","Jiuxiang Gu"],"abstract":"Chain-of-Thought (CoT) reasoning has become a powerful framework for improving complex problem-solving capabilities in Multimodal Large Language Models (MLLMs). However, the verbose nature of textual reasoning introduces significant inefficiencies. In this work, we propose $\\textbf{Heima}$ (as hidden llama), an efficient reasoning framework that leverages reasoning CoTs at hidden latent space. We design the Heima Encoder to condense each intermediate CoT into a compact, higher-level hidden representation using a single thinking token, effectively minimizing verbosity and reducing the overall number of tokens required during the reasoning process. Meanwhile, we design corresponding Heima Decoder with traditional Large Language Models (LLMs) to adaptively interpret the hidden representations into variable-length textual sequence, reconstructing reasoning processes that closely resemble the original CoTs. Experimental results across diverse reasoning MLLM benchmarks demonstrate that Heima model achieves higher generation efficiency while maintaining or even better zero-shot task accuracy. Moreover, the effective reconstruction of multimodal reasoning processes with Heima Decoder validates both the robustness and interpretability of our approach.","url_abs":"https://arxiv.org/abs/2501.19201v1","url_pdf":"https://arxiv.org/pdf/2501.19201v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-reasoning-with-hidden-thinking","repo_url":"https://github.com/shawnricecake/heima","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"multimodal-reasoning","task_name":"Multimodal Reasoning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2501.19201","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2501.19201"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shawnricecake/heima","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":3,"ran_fixture":1},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"054f0c94c7ad7580","entry":"load_from_json","repo":"shawnricecake/Heima","repo_kind":"official","path":"heima/main_python/1_1-organize_dataset-num_thinking_tokens-fix_num.py","file_url":"https://github.com/shawnricecake/Heima/blob/HEAD/heima/main_python/1_1-organize_dataset-num_thinking_tokens-fix_num.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"054f0c94c7ad7580"}},{"code_sha256_prefix":"396e58efde50875b","entry":"extract_sections","repo":"shawnricecake/Heima","repo_kind":"official","path":"heima/main_python/1_1-organize_dataset-num_thinking_tokens-fix_num.py","file_url":"https://github.com/shawnricecake/Heima/blob/HEAD/heima/main_python/1_1-organize_dataset-num_thinking_tokens-fix_num.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"396e58efde50875b"}},{"code_sha256_prefix":"076a1a6c927df3dc","entry":"get_middle_sublist","repo":"shawnricecake/Heima","repo_kind":"official","path":"heima/main_python/1_1-organize_dataset-num_thinking_tokens-fix_num.py","file_url":"https://github.com/shawnricecake/Heima/blob/HEAD/heima/main_python/1_1-organize_dataset-num_thinking_tokens-fix_num.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"076a1a6c927df3dc"}},{"code_sha256_prefix":"bbaa745857cf091e","entry":"load_from_json","repo":"shawnricecake/Heima","repo_kind":"official","path":"heima/main_python/5-demo-decode-pure_llm_decoder_lora-split_3_stages.py","file_url":"https://github.com/shawnricecake/Heima/blob/HEAD/heima/main_python/5-demo-decode-pure_llm_decoder_lora-split_3_stages.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bbaa745857cf091e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}