{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/frictional-agent-alignment-framework-slow","title":"Frictional Agent Alignment Framework: Slow Down and Don't Break Things","arxiv_id":"2505.19428","date":"2025-05-26","proceeding":null,"authors":["Abhijnan Nath","Carine Graff","Andrei Bachinin","Nikhil Krishnaswamy"],"abstract":"AI support of collaborative interactions entails mediating potential misalignment between interlocutor beliefs. Common preference alignment methods like DPO excel in static settings, but struggle in dynamic collaborative tasks where the explicit signals of interlocutor beliefs are sparse and skewed. We propose the Frictional Agent Alignment Framework (FAAF), to generate precise, context-aware \"friction\" that prompts for deliberation and re-examination of existing evidence. FAAF's two-player objective decouples from data skew: a frictive-state policy identifies belief misalignments, while an intervention policy crafts collaborator-preferred responses. We derive an analytical solution to this objective, enabling training a single policy via a simple supervised loss. Experiments on three benchmarks show FAAF outperforms competitors in producing concise, interpretable friction and in OOD generalization. By aligning LLMs to act as adaptive \"thought partners\" -- not passive responders -- FAAF advances scalable, dynamic human-AI collaboration. Our code and data can be found at https://github.com/csu-signal/FAAF_ACL.","url_abs":"https://arxiv.org/abs/2505.19428v1","url_pdf":"https://arxiv.org/pdf/2505.19428v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"frictional-agent-alignment-framework-slow","repo_url":"https://github.com/csu-signal/faaf_acl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"friction","task_name":"Friction"}],"methods":[{"method_slug":"dpo","method_name":"DPO"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2505.19428","atlas_url":"https://app.syntology.ai/?focus=2505.19428","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.19428"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/csu-signal/FAAF_ACL","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":5,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"758b839d26630385","entry":"transform_and_assign_preference_deli","repo":"csu-signal/FAAF_ACL","repo_kind":"official","path":"ppo_baseline_training.py","file_url":"https://github.com/csu-signal/FAAF_ACL/blob/HEAD/ppo_baseline_training.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"758b839d26630385"}},{"code_sha256_prefix":"8a1c09011c8b16e7","entry":"transform_and_assign_preferences","repo":"csu-signal/FAAF_ACL","repo_kind":"official","path":"ppo_baseline_training.py","file_url":"https://github.com/csu-signal/FAAF_ACL/blob/HEAD/ppo_baseline_training.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8a1c09011c8b16e7"}},{"code_sha256_prefix":"0b5c8529d88e68f1","entry":"transform_and_assign_preferences_2","repo":"csu-signal/FAAF_ACL","repo_kind":"official","path":"ppo_baseline_training.py","file_url":"https://github.com/csu-signal/FAAF_ACL/blob/HEAD/ppo_baseline_training.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0b5c8529d88e68f1"}},{"code_sha256_prefix":"29778de54fcf7f64","entry":"transform_and_assign_preferences_deli","repo":"csu-signal/FAAF_ACL","repo_kind":"official","path":"opt_reward_modeling.py","file_url":"https://github.com/csu-signal/FAAF_ACL/blob/HEAD/opt_reward_modeling.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"29778de54fcf7f64"}},{"code_sha256_prefix":"355f8800dc1410a1","entry":"transform_and_assign_preferences_wtd","repo":"csu-signal/FAAF_ACL","repo_kind":"official","path":"opt_reward_modeling.py","file_url":"https://github.com/csu-signal/FAAF_ACL/blob/HEAD/opt_reward_modeling.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"355f8800dc1410a1"}},{"code_sha256_prefix":"962f0e9962c9f8a8","entry":"format_evaluation_prompt","repo":"csu-signal/FAAF_ACL","repo_kind":"official","path":"llm_judge_evals.py","file_url":"https://github.com/csu-signal/FAAF_ACL/blob/HEAD/llm_judge_evals.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"962f0e9962c9f8a8"}},{"code_sha256_prefix":"5212f303a42f980a","entry":"preprocess_function","repo":"csu-signal/FAAF_ACL","repo_kind":"official","path":"opt_reward_modeling.py","file_url":"https://github.com/csu-signal/FAAF_ACL/blob/HEAD/opt_reward_modeling.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5212f303a42f980a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}