{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-01663","title":"Towards Efficient and Expressive Offline RL via Flow-Anchored Noise-conditioned Q-Learning","arxiv_id":"2605.01663","date":"2026-05-03","proceeding":"ICML","authors":["Sungyoung Lee","Dohyeong Kim","Eshan Balachandar","Zelal Su Mustafaoglu","Keshav Pingali"],"abstract":"We propose Flow-Anchored Noise-conditioned Q-Learning (FAN), a highly efficient and high-performing offline reinforcement learning (RL) algorithm. Recent work has shown that expressive flow policies and distributional critics improve offline RL performance, but at a high computational cost. Specifically, flow policies require iterative sampling to produce a single action, and distributional critics require computation over multiple samples (e.g., quantiles) to estimate value. To address these inefficiencies while maintaining high performance, we introduce FAN. Our method employs a behavior regularization technique that uses a single flow policy iteration and requires a single Gaussian noise sample for distributional critics. Our theoretical analysis of convergence and performance bounds demonstrates that these simplifications not only improve efficiency but also lead to superior task performance. Experiments on robotic manipulation and locomotion tasks demonstrate that FAN achieves state-of-the-art performance while significantly reducing both training and inference runtimes. We release our code at https://github.com/brianlsy98/FAN.","url_abs":"https://arxiv.org/abs/2605.01663","url_pdf":"https://arxiv.org/pdf/2605.01663","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2605.01663","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.01663"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/brianlsy98/FAN","reach":null}],"summary":{"ran":4,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c5ada5c7d43ea8f4","entry":"ActorVectorField","repo":"brianlsy98/FAN","repo_kind":"found_in_text","path":"agents/fan.py","file_url":"https://github.com/brianlsy98/FAN/blob/HEAD/agents/fan.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c5ada5c7d43ea8f4"}},{"code_sha256_prefix":"5c2823b42170fed6","entry":"ImpalaEncoder","repo":"brianlsy98/FAN","repo_kind":"found_in_text","path":"agents/fan.py","file_url":"https://github.com/brianlsy98/FAN/blob/HEAD/agents/fan.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5c2823b42170fed6"}},{"code_sha256_prefix":"c10b91c12ad9a306","entry":"MLP","repo":"brianlsy98/FAN","repo_kind":"found_in_text","path":"agents/fan.py","file_url":"https://github.com/brianlsy98/FAN/blob/HEAD/agents/fan.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c10b91c12ad9a306"}},{"code_sha256_prefix":"9235803e8ce9944a","entry":"Value","repo":"brianlsy98/FAN","repo_kind":"found_in_text","path":"agents/fan.py","file_url":"https://github.com/brianlsy98/FAN/blob/HEAD/agents/fan.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9235803e8ce9944a"}},{"code_sha256_prefix":"2f9a4dc27b5290d0","entry":"FANAgent","repo":"brianlsy98/FAN","repo_kind":"found_in_text","path":"agents/fan.py","file_url":"https://github.com/brianlsy98/FAN/blob/HEAD/agents/fan.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2f9a4dc27b5290d0"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}