{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-23940","title":"Residual Drift Dominates Contradiction in Multi-Turn Constraint Reasoning","arxiv_id":"2605.23940","date":"2026-04-28","proceeding":null,"authors":["Sebastien Kawada"],"abstract":"How do multi-turn reasoning systems fail? The expected answer is logical contradiction, in which the system's maintained state becomes unsatisfiable. We show that the dominant mode is instead satisfiable drift, where the internal state stays consistent while the returned answer silently violates prior commitments. We build DRIFT-Bench (Decomposing Reasoning Into Failure Types), a solver-instrumented benchmark of 816 test problems across three constraint domains, and evaluate four methods on it across four open-weight models (8B-120B parameters). MUS-Repair, which feeds minimal unsatisfiable subsets back to the generator, is strongest in every setting (+1.8 to +15.0 pp over the best non-MUS baseline). But the central finding is what repair leaves behind. After structured feedback, models rarely contradict themselves. They forget. Residual errors are 98-100% satisfiable drift across all settings, while contradiction drops to near zero. Reliable multi-turn systems must separately validate that the returned answer respects the maintained state. Code is available at https://github.com/kaons-research/drift-bench.","url_abs":"https://arxiv.org/abs/2605.23940","url_pdf":"https://arxiv.org/pdf/2605.23940","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2605.23940","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.23940"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/kaons-research/drift-bench","reach":null}],"summary":{"ran":17,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":19,"ran":17,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6b3409ba60d6f4b6","entry":"build_answer_retry_messages","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/prompts.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/prompts.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6b3409ba60d6f4b6"}},{"code_sha256_prefix":"a9dc69f03fd54a01","entry":"build_truncation_retry_messages","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/prompts.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/prompts.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a9dc69f03fd54a01"}},{"code_sha256_prefix":"e08375efdbb3fce2","entry":"build_turn_messages","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/prompts.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/prompts.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e08375efdbb3fce2"}},{"code_sha256_prefix":"87aa5907f0cb48ee","entry":"build_user_message","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/generate_problems.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/generate_problems.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"87aa5907f0cb48ee"}},{"code_sha256_prefix":"421bced5ee970b0e","entry":"canonicalize_constraint","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/z3_checker.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/z3_checker.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"421bced5ee970b0e"}},{"code_sha256_prefix":"a23de49f4a4e2ca2","entry":"estimate_tokens_from_text","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/run_experiment.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/run_experiment.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a23de49f4a4e2ca2"}},{"code_sha256_prefix":"5727f4a3df316a5a","entry":"extract_json_block","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/extraction.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/extraction.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5727f4a3df316a5a"}},{"code_sha256_prefix":"fa850eb8d5f90663","entry":"ledger_token_count","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/run_experiment.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/run_experiment.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fa850eb8d5f90663"}},{"code_sha256_prefix":"e1dd1a7c6320d923","entry":"load_problem_gold","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/analyze.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/analyze.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e1dd1a7c6320d923"}},{"code_sha256_prefix":"4737d24ad6135c21","entry":"load_problems","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/run_experiment.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/run_experiment.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4737d24ad6135c21"}},{"code_sha256_prefix":"9f1ee38270e3cbda","entry":"load_rows","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/analyze.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/analyze.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f1ee38270e3cbda"}},{"code_sha256_prefix":"8bead903b69ffed0","entry":"ordered_unique","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/repair_policy.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/repair_policy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8bead903b69ffed0"}},{"code_sha256_prefix":"2added5f4145a1d2","entry":"pick_distinct_pair","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/generate_problems.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/generate_problems.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2added5f4145a1d2"}},{"code_sha256_prefix":"1a41a1bb65acebd1","entry":"policy_uses_sat_triggers","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/repair_policy.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/repair_policy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1a41a1bb65acebd1"}},{"code_sha256_prefix":"d33b30864c53fb64","entry":"sample_seating_constraint","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/generate_problems.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/generate_problems.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d33b30864c53fb64"}},{"code_sha256_prefix":"82a316cc22bc1015","entry":"table_columns","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/analyze.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/analyze.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"82a316cc22bc1015"}},{"code_sha256_prefix":"26f1b08bdc9578f4","entry":"unique_trigger_codes","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/repair_policy.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/repair_policy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"26f1b08bdc9578f4"}},{"code_sha256_prefix":"2859b0d4274869b4","entry":"extract_answer","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/extraction.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/extraction.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2859b0d4274869b4"}},{"code_sha256_prefix":"94d9078851c10b86","entry":"extract_answer_with_retry","repo":"kaons-research/drift-bench","repo_kind":"found_in_text","path":"src/extraction.py","file_url":"https://github.com/kaons-research/drift-bench/blob/HEAD/src/extraction.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"94d9078851c10b86"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}