{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generative-flow-networks-as-entropy","title":"Generative Flow Networks as Entropy-Regularized RL","arxiv_id":"2310.12934","date":"2023-10-19","proceeding":null,"authors":["Daniil Tiapkin","Nikita Morozov","Alexey Naumov","Dmitry Vetrov"],"abstract":"The recently proposed generative flow networks (GFlowNets) are a method of training a policy to sample compositional discrete objects with probabilities proportional to a given reward via a sequence of actions. GFlowNets exploit the sequential nature of the problem, drawing parallels with reinforcement learning (RL). Our work extends the connection between RL and GFlowNets to a general case. We demonstrate how the task of learning a generative flow network can be efficiently redefined as an entropy-regularized RL problem with a specific reward and regularizer structure. Furthermore, we illustrate the practical efficiency of this reformulation by applying standard soft RL algorithms to GFlowNet training across several probabilistic modeling tasks. Contrary to previously reported results, we show that entropic RL approaches can be competitive against established GFlowNet training methods. This perspective opens a direct path for integrating RL principles into the realm of generative flow networks.","url_abs":"https://arxiv.org/abs/2310.12934v3","url_pdf":"https://arxiv.org/pdf/2310.12934v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generative-flow-networks-as-entropy","repo_url":"https://github.com/d-tiapkin/gflownet-rl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.12934","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.12934"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/d-tiapkin/gflownet-rl","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"unverified":5},"by_repo_kind":{"official":{"samples":7,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6b452c8d0cc45c99","entry":"edge_match","repo":"d-tiapkin/gflownet-rl","repo_kind":"official","path":"mols/mol_mdp_ext.py","file_url":"https://github.com/d-tiapkin/gflownet-rl/blob/HEAD/mols/mol_mdp_ext.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6b452c8d0cc45c99"}},{"code_sha256_prefix":"514f704ea4cae3e4","entry":"node_match","repo":"d-tiapkin/gflownet-rl","repo_kind":"official","path":"mols/mol_mdp_ext.py","file_url":"https://github.com/d-tiapkin/gflownet-rl/blob/HEAD/mols/mol_mdp_ext.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"514f704ea4cae3e4"}},{"code_sha256_prefix":"c8525326061a88fd","entry":"detailed_balance_loss","repo":"d-tiapkin/gflownet-rl","repo_kind":"official","path":"mols/gflownet.py","file_url":"https://github.com/d-tiapkin/gflownet-rl/blob/HEAD/mols/gflownet.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c8525326061a88fd"}},{"code_sha256_prefix":"f763f98f79152e0e","entry":"mol2graph","repo":"d-tiapkin/gflownet-rl","repo_kind":"official","path":"mols/model_block.py","file_url":"https://github.com/d-tiapkin/gflownet-rl/blob/HEAD/mols/model_block.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f763f98f79152e0e"}},{"code_sha256_prefix":"06477cbf36090384","entry":"mols2batch","repo":"d-tiapkin/gflownet-rl","repo_kind":"official","path":"mols/model_atom.py","file_url":"https://github.com/d-tiapkin/gflownet-rl/blob/HEAD/mols/model_atom.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"06477cbf36090384"}},{"code_sha256_prefix":"f99fcc80e4f63a87","entry":"tb_lambda_loss","repo":"d-tiapkin/gflownet-rl","repo_kind":"official","path":"mols/gflownet.py","file_url":"https://github.com/d-tiapkin/gflownet-rl/blob/HEAD/mols/gflownet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f99fcc80e4f63a87"}},{"code_sha256_prefix":"afa974460f51f1a5","entry":"trajectory_balance_loss","repo":"d-tiapkin/gflownet-rl","repo_kind":"official","path":"mols/gflownet.py","file_url":"https://github.com/d-tiapkin/gflownet-rl/blob/HEAD/mols/gflownet.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"afa974460f51f1a5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}