{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/optimizing-backward-policies-in-gflownets-via","title":"Optimizing Backward Policies in GFlowNets via Trajectory Likelihood Maximization","arxiv_id":"2410.15474","date":"2024-10-20","proceeding":null,"authors":["Timofei Gritsaev","Nikita Morozov","Sergey Samsonov","Daniil Tiapkin"],"abstract":"Generative Flow Networks (GFlowNets) are a family of generative models that learn to sample objects with probabilities proportional to a given reward function. The key concept behind GFlowNets is the use of two stochastic policies: a forward policy, which incrementally constructs compositional objects, and a backward policy, which sequentially deconstructs them. Recent results show a close relationship between GFlowNet training and entropy-regularized reinforcement learning (RL) problems with a particular reward design. However, this connection applies only in the setting of a fixed backward policy, which might be a significant limitation. As a remedy to this problem, we introduce a simple backward policy optimization algorithm that involves direct maximization of the value function in an entropy-regularized Markov Decision Process (MDP) over intermediate rewards. We provide an extensive experimental evaluation of the proposed approach across various benchmarks in combination with both RL and GFlowNet algorithms and demonstrate its faster convergence and mode discovery in complex environments.","url_abs":"https://arxiv.org/abs/2410.15474v1","url_pdf":"https://arxiv.org/pdf/2410.15474v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"optimizing-backward-policies-in-gflownets-via","repo_url":"https://github.com/tgritsaev/gflownet-tlm","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2410.15474","atlas_url":"https://app.syntology.ai/?focus=2410.15474","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.15474"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/tgritsaev/gflownet-tlm","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"89bd3b8341e51ebc","entry":"M_distance","repo":"tgritsaev/gflownet-tlm","repo_kind":"official","path":"bitseq/utils.py","file_url":"https://github.com/tgritsaev/gflownet-tlm/blob/HEAD/bitseq/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"89bd3b8341e51ebc"}},{"code_sha256_prefix":"b2afd99dae98ed88","entry":"construct_M","repo":"tgritsaev/gflownet-tlm","repo_kind":"official","path":"bitseq/utils.py","file_url":"https://github.com/tgritsaev/gflownet-tlm/blob/HEAD/bitseq/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b2afd99dae98ed88"}},{"code_sha256_prefix":"94f929faf5e1b4e9","entry":"distance","repo":"tgritsaev/gflownet-tlm","repo_kind":"official","path":"bitseq/utils.py","file_url":"https://github.com/tgritsaev/gflownet-tlm/blob/HEAD/bitseq/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"94f929faf5e1b4e9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}