{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2608-24350","title":"FARCA: Fact-Aligned Reliability-Aware Credit Assignment for Reinforcement Learning with Factual Supervision","arxiv_id":"2608.24350","date":"2026-08-25","proceeding":null,"authors":["Qiming Xie","Wenjie Zheng","Xiangqing Shen","Rui Xia"],"abstract":"To reduce the hallucination risk caused by outcome-driven rewards in large language models trained through reinforcement learning with verifiable rewards, existing mitigation approaches introduce process-level factual supervision. However, due to coarse-grained aggregation of factual signals and the lack of reliability assessment for these signals, they create a mismatch between fact verification and policy updates. We term this noisy factual credit assignment and decompose it into two aspects: credit localization ambiguity and credit reliability ambiguity. To address these issues, we propose FARCA (Fact-Aligned Reliability-Aware Credit Assignment), a policy optimization framework that transforms factual supervision into localized, reliability-weighted token-level training signals. FARCA achieves fine-grained credit localization by aligning the granularity of fact verification with that of policy updates. It further introduces counterfactual evidence attribution, which uses the dependence of a factual judgment on key evidence as an empirical proxy for verification reliability to compute reliability weights. These weights modulate factual rewards and local policy advantages, reducing the influence of potentially unreliable signals on policy optimization. Experiments across different models and multiple factual reasoning benchmarks show that FARCA significantly improves model factuality while preserving general reasoning capabilities.","url_abs":"https://arxiv.org/abs/2608.24350","url_pdf":"https://arxiv.org/pdf/2608.24350","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2608.24350","atlas_url":"https://app.syntology.ai/?focus=2608.24350","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2608.24350"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/verl-project/verl","reach":null}],"summary":{"ran_draft_wrong":3,"ran":1,"ran_violates":1,"unverified":7},"by_repo_kind":{"found_in_text":{"samples":12,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"45205d24c28c95d4","entry":"generate_config_from_args","repo":"verl-project/verl","repo_kind":"found_in_text","path":"verl/model_merger/base_model_merger.py","file_url":"https://github.com/verl-project/verl/blob/HEAD/verl/model_merger/base_model_merger.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"45205d24c28c95d4"}},{"code_sha256_prefix":"94209884dcb2308e","entry":"get_dynamic_pipeline_shards","repo":"verl-project/verl","repo_kind":"found_in_text","path":"verl/model_merger/megatron_model_merger.py","file_url":"https://github.com/verl-project/verl/blob/HEAD/verl/model_merger/megatron_model_merger.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"94209884dcb2308e"}},{"code_sha256_prefix":"735e93af30b15fa9","entry":"is_ipv4","repo":"verl-project/verl","repo_kind":"found_in_text","path":"verl/utils/net_utils.py","file_url":"https://github.com/verl-project/verl/blob/HEAD/verl/utils/net_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"735e93af30b15fa9"}},{"code_sha256_prefix":"c177993b39fcc0e5","entry":"merge_non_dtensor_shards","repo":"verl-project/verl","repo_kind":"found_in_text","path":"verl/model_merger/fsdp_model_merger.py","file_url":"https://github.com/verl-project/verl/blob/HEAD/verl/model_merger/fsdp_model_merger.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c177993b39fcc0e5"}},{"code_sha256_prefix":"83b1686d7ac03c21","entry":"unpad_dataproto","repo":"verl-project/verl","repo_kind":"found_in_text","path":"verl/protocol.py","file_url":"https://github.com/verl-project/verl/blob/HEAD/verl/protocol.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"83b1686d7ac03c21"}},{"code_sha256_prefix":"687521ab4cf0bf52","entry":"get_generation_config","repo":"verl-project/verl","repo_kind":"found_in_text","path":"verl/utils/model.py","file_url":"https://github.com/verl-project/verl/blob/HEAD/verl/utils/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"687521ab4cf0bf52"}},{"code_sha256_prefix":"3033106901336d62","entry":"get_huggingface_actor_config","repo":"verl-project/verl","repo_kind":"found_in_text","path":"verl/utils/model.py","file_url":"https://github.com/verl-project/verl/blob/HEAD/verl/utils/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3033106901336d62"}},{"code_sha256_prefix":"acf240079d434746","entry":"get_weight_loader","repo":"verl-project/verl","repo_kind":"found_in_text","path":"verl/models/weight_loader_registry.py","file_url":"https://github.com/verl-project/verl/blob/HEAD/verl/models/weight_loader_registry.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"acf240079d434746"}},{"code_sha256_prefix":"ada2d6cf9ca9ee89","entry":"get_weight_saver","repo":"verl-project/verl","repo_kind":"found_in_text","path":"verl/models/weight_loader_registry.py","file_url":"https://github.com/verl-project/verl/blob/HEAD/verl/models/weight_loader_registry.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ada2d6cf9ca9ee89"}},{"code_sha256_prefix":"705afe175ad8e404","entry":"is_ipv6","repo":"verl-project/verl","repo_kind":"found_in_text","path":"verl/utils/net_utils.py","file_url":"https://github.com/verl-project/verl/blob/HEAD/verl/utils/net_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"705afe175ad8e404"}},{"code_sha256_prefix":"72b5e8089e8c68a6","entry":"is_valid_ipv6_address","repo":"verl-project/verl","repo_kind":"found_in_text","path":"verl/utils/net_utils.py","file_url":"https://github.com/verl-project/verl/blob/HEAD/verl/utils/net_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"72b5e8089e8c68a6"}},{"code_sha256_prefix":"3b15e2ac7497c441","entry":"squeeze","repo":"verl-project/verl","repo_kind":"found_in_text","path":"verl/utils/model.py","file_url":"https://github.com/verl-project/verl/blob/HEAD/verl/utils/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3b15e2ac7497c441"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":{"kind":"leaderboard_placements","source":"Syntology's leaderboard-shaped extractor over the paper's own arXiv-HTML tables: a model pointed at a cell, the number was read from that cell and checked against the board's metric, dataset, split and scale, and an independent check accepted the entry; not reviewed by the paper's authors or the archive's editors","extractor_model":"global.anthropic.claude-sonnet-4-5-20250929-v1:0","verifier_model":null,"prompt_sha":"fa63d4bb9d755694","coverage":{"sentence":"Syntology has checked 6,885 of the 9,623 papers on this site that are newer than the archive; results from the others appear after they are checked.","papers_newer_than_archive":9623,"papers_checked":6885},"entries":[],"not_placed":{"boards":0,"rejected_by_independent_check":0,"refused_by_a_rule":0,"check_did_not_answer":0,"proposed_without_a_cell":0,"declined_by_site":0}}}