{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2606-00869","title":"Enhancing LLM Metacognition via Cognitive Pairwise Training","arxiv_id":"2606.00869","date":"2026-05-30","proceeding":null,"authors":["Weitao Li","Hao Zhou","Xuanyu Lei","Fandong Meng","Yuanhang Liu","Jingyi Ren","Ante Wang","Xiaolong Wang","Yuanchi Zhang","Fuwen Luo","Guangwen Yang","Lin Gan","Weizhi Ma","Yang Liu"],"abstract":"Reinforcement learning with verifiable rewards (RLVR) has become central to LLM reasoning, but its outcome-level rewards can make models more willing to give confident answers when evidence or reasoning is unreliable. Existing SFT or RL methods mainly teach LLMs to refuse or express uncertainty at the response level, which can overfit abstention behavior rather than improve reasoning reliability. To address this limitation, we propose Cognitive Pairwise Training (CPT), a cognitive mid-training alignment stage that turns pairwise comparisons over reasoning traces into a reusable alignment signal. By learning to distinguish trustworthy from flawed reasoning, CPT encourages the model to internalize a reasoning-quality discrimination boundary rather than memorize surface refusal patterns. Across five model scales and three model families, CPT improves the reasoning--metacognition trade-off. At 14B, CPT+RL outperforms the standard SFT+RL pipeline by +2.2 math-average points and +5.6 abstention-F1 points. Further analyses show that CPT improves trace quality and exhibits strong robustness and scalability across evaluation and training settings. Code and models are released at https://github.com/Tsinghua-dhy/CPT.","url_abs":"https://arxiv.org/abs/2606.00869","url_pdf":"https://arxiv.org/pdf/2606.00869","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2606.00869","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2606.00869"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/Tsinghua-dhy/CPT","reach":null}],"summary":{"ran":21,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":23,"ran":21,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"74944fb1c9af1d08","entry":"build_prompt","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"data_construction/4_build_sft/build_sft_dataset.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/data_construction/4_build_sft/build_sft_dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"74944fb1c9af1d08"}},{"code_sha256_prefix":"b9d58aa1639207a0","entry":"detect_model_type","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"eval/abstention/eval_abstention.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/eval/abstention/eval_abstention.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b9d58aa1639207a0"}},{"code_sha256_prefix":"0f9fe639bed6c737","entry":"extract_answer_math","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"data_construction/1_rollout/rollout.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/data_construction/1_rollout/rollout.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0f9fe639bed6c737"}},{"code_sha256_prefix":"ffcf6f704a28bb3b","entry":"extract_answer_math","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"eval/math/eval_math.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/eval/math/eval_math.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ffcf6f704a28bb3b"}},{"code_sha256_prefix":"5a65ce548f2db502","entry":"extract_boxed_content","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"data_construction/3_judge/judge_pairs.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/data_construction/3_judge/judge_pairs.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5a65ce548f2db502"}},{"code_sha256_prefix":"e5cec7c5865db846","entry":"get_all_highest_confidence_answers","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"data_construction/4_build_sft/build_sft_dataset.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/data_construction/4_build_sft/build_sft_dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e5cec7c5865db846"}},{"code_sha256_prefix":"faa29ea87424c295","entry":"get_model_size","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"data_construction/2_pair/build_pairs.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/data_construction/2_pair/build_pairs.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"faa29ea87424c295"}},{"code_sha256_prefix":"04002d844af70af7","entry":"get_prompt","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"eval/utils/judge_math_answer_gpt.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/eval/utils/judge_math_answer_gpt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"04002d844af70af7"}},{"code_sha256_prefix":"a8eed4c1f2f2831b","entry":"get_prompt","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"eval/utils/math_equal.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/eval/utils/math_equal.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a8eed4c1f2f2831b"}},{"code_sha256_prefix":"41a2c29e0316585f","entry":"get_task_type","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"eval/rag/eval_rag_conflicts.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/eval/rag/eval_rag_conflicts.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"41a2c29e0316585f"}},{"code_sha256_prefix":"5d63c771657aa174","entry":"is_instruct_model","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"eval/math/eval_math.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/eval/math/eval_math.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5d63c771657aa174"}},{"code_sha256_prefix":"a2cb58cb395f9e4b","entry":"is_instruct_model","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"eval/rag/eval_rag_conflicts.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/eval/rag/eval_rag_conflicts.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a2cb58cb395f9e4b"}},{"code_sha256_prefix":"4afecbe3b89e8ab8","entry":"load_all_questions","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"data_construction/2_pair/build_pairs.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/data_construction/2_pair/build_pairs.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4afecbe3b89e8ab8"}},{"code_sha256_prefix":"bc1311386b177375","entry":"normalize_judgment","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"data_construction/3_judge/judge_pairs.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/data_construction/3_judge/judge_pairs.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bc1311386b177375"}},{"code_sha256_prefix":"d4cd0d203051a63a","entry":"normalize_rollout","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"data_construction/2_pair/build_pairs.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/data_construction/2_pair/build_pairs.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d4cd0d203051a63a"}},{"code_sha256_prefix":"ce2f3b3913ae8706","entry":"parse_judge_output","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"data_construction/3_judge/judge_pairs.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/data_construction/3_judge/judge_pairs.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ce2f3b3913ae8706"}},{"code_sha256_prefix":"13430c1445a0990c","entry":"process_text","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"data_construction/1_rollout/rollout.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/data_construction/1_rollout/rollout.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"13430c1445a0990c"}},{"code_sha256_prefix":"7f580ad4df20e7bd","entry":"process_text","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"eval/math/eval_math.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/eval/math/eval_math.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7f580ad4df20e7bd"}},{"code_sha256_prefix":"14c46927d4562367","entry":"split_consensus_data","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"data_construction/4_build_sft/split_consensus.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/data_construction/4_build_sft/split_consensus.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"14c46927d4562367"}},{"code_sha256_prefix":"6f4528b6a787c430","entry":"validate_datasets","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"eval/abstention/eval_abstention.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/eval/abstention/eval_abstention.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6f4528b6a787c430"}},{"code_sha256_prefix":"fbe2f6cfb047b17e","entry":"validate_prompt_styles","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"eval/abstention/eval_abstention.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/eval/abstention/eval_abstention.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fbe2f6cfb047b17e"}},{"code_sha256_prefix":"990b7ec66e5cee70","entry":"compute_math_score","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"eval/abstention/correctness_evaluator.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/eval/abstention/correctness_evaluator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"990b7ec66e5cee70"}},{"code_sha256_prefix":"ba60eead339e8791","entry":"process_text","repo":"Tsinghua-dhy/CPT","repo_kind":"found_in_text","path":"eval/rag/eval_rag_conflicts.py","file_url":"https://github.com/Tsinghua-dhy/CPT/blob/HEAD/eval/rag/eval_rag_conflicts.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ba60eead339e8791"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}