{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2603-14245","title":"GoldenStart: Q-Guided Priors and Entropy Control for Distilling Flow Policies","arxiv_id":"2603.14245","date":"2026-03-15","proceeding":null,"authors":["He Zhang","Ying Sun","Hui Xiong"],"abstract":"Flow-matching policies hold great promise for reinforcement learning (RL) by capturing complex, multi-modal action distributions. However, their practical application is often hindered by prohibitive inference latency and ineffective online exploration. Although recent works have employed one-step distillation for fast inference, the structure of the initial noise distribution remains an overlooked factor that presents significant untapped potential. This overlooked factor, along with the challenge of controlling policy stochasticity, constitutes two critical areas for advancing distilled flow-matching policies. To overcome these limitations, we propose GoldenStart (GSFlow), a policy distillation method with Q-guided priors and explicit entropy control. Instead of initializing generation from uninformed noise, we introduce a Q-guided prior modeled by a conditional VAE. This state-conditioned prior repositions the starting points of the one-step generation process into high-Q regions, effectively providing a \"golden start\" that shortcuts the policy to promising actions. Furthermore, for effective online exploration, we enable our distilled actor to output a stochastic distribution instead of a deterministic point. This is governed by entropy regularization, allowing the policy to shift from pure exploitation to principled exploration. Our integrated framework demonstrates that by designing the generative startpoint and explicitly controlling policy entropy, it is possible to achieve efficient and exploratory policies, bridging the generative models and the practical actor-critic methods. We conduct extensive experiments on offline and online continuous control benchmarks, where our method significantly outperforms prior state-of-the-art approaches. Code will be available at https://github.com/ZhHe11/GSFlow-RL.","url_abs":"https://arxiv.org/abs/2603.14245","url_pdf":"https://arxiv.org/pdf/2603.14245","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2603.14245","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2603.14245"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/ZhHe11/GSFlow-RL","reach":null}],"summary":{"unverified":7},"by_repo_kind":{"found_in_text":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9ee52fbef6352c3e","entry":"ActorVectorField","repo":"ZhHe11/GSFlow-RL","repo_kind":"found_in_text","path":"agents/gsflow.py","file_url":"https://github.com/ZhHe11/GSFlow-RL/blob/HEAD/agents/gsflow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9ee52fbef6352c3e"}},{"code_sha256_prefix":"cde44c8ebd779553","entry":"Actor_Noise","repo":"ZhHe11/GSFlow-RL","repo_kind":"found_in_text","path":"agents/gsflow.py","file_url":"https://github.com/ZhHe11/GSFlow-RL/blob/HEAD/agents/gsflow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cde44c8ebd779553"}},{"code_sha256_prefix":"5d8e3e4e78530585","entry":"Decoder","repo":"ZhHe11/GSFlow-RL","repo_kind":"found_in_text","path":"agents/gsflow.py","file_url":"https://github.com/ZhHe11/GSFlow-RL/blob/HEAD/agents/gsflow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5d8e3e4e78530585"}},{"code_sha256_prefix":"f251b0f125d7f21d","entry":"Encoder","repo":"ZhHe11/GSFlow-RL","repo_kind":"found_in_text","path":"agents/gsflow.py","file_url":"https://github.com/ZhHe11/GSFlow-RL/blob/HEAD/agents/gsflow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f251b0f125d7f21d"}},{"code_sha256_prefix":"dcc34d5e2db05b57","entry":"GSflow","repo":"ZhHe11/GSFlow-RL","repo_kind":"found_in_text","path":"agents/gsflow.py","file_url":"https://github.com/ZhHe11/GSFlow-RL/blob/HEAD/agents/gsflow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dcc34d5e2db05b57"}},{"code_sha256_prefix":"f3b51465ed3c7cd8","entry":"LogParam","repo":"ZhHe11/GSFlow-RL","repo_kind":"found_in_text","path":"agents/gsflow.py","file_url":"https://github.com/ZhHe11/GSFlow-RL/blob/HEAD/agents/gsflow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f3b51465ed3c7cd8"}},{"code_sha256_prefix":"131d043e72cb8f14","entry":"Value","repo":"ZhHe11/GSFlow-RL","repo_kind":"found_in_text","path":"agents/gsflow.py","file_url":"https://github.com/ZhHe11/GSFlow-RL/blob/HEAD/agents/gsflow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"131d043e72cb8f14"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}