{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/leveraging-recursive-gumbel-max-trick-for","title":"Leveraging Recursive Gumbel-Max Trick for Approximate Inference in Combinatorial Spaces","arxiv_id":"2110.15072","date":"2021-10-28","proceeding":"NeurIPS 2021 12","authors":["Kirill Struminsky","Artyom Gadetsky","Denis Rakitin","Danil Karpushkin","Dmitry Vetrov"],"abstract":"Structured latent variables allow incorporating meaningful prior knowledge into deep learning models. However, learning with such variables remains challenging because of their discrete nature. Nowadays, the standard learning approach is to define a latent variable as a perturbed algorithm output and to use a differentiable surrogate for training. In general, the surrogate puts additional constraints on the model and inevitably leads to biased gradients. To alleviate these shortcomings, we extend the Gumbel-Max trick to define distributions over structured domains. We avoid the differentiable surrogates by leveraging the score function estimators for optimization. In particular, we highlight a family of recursive algorithms with a common feature we call stochastic invariant. The feature allows us to construct reliable gradient estimates and control variates without additional constraints on the model. In our experiments, we consider various structured latent variable models and achieve results competitive with relaxation-based counterparts.","url_abs":"https://arxiv.org/abs/2110.15072v1","url_pdf":"https://arxiv.org/pdf/2110.15072v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"leveraging-recursive-gumbel-max-trick-for","repo_url":"https://github.com/RakitinDen/pytorch-recursive-gumbel-max-trick","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2110.15072","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.15072"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/RakitinDen/pytorch-recursive-gumbel-max-trick","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":10},"by_repo_kind":{"official":{"samples":10,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"991472e4dbd1145d","entry":"E_reinforce","repo":"RakitinDen/pytorch-recursive-gumbel-max-trick","repo_kind":"official","path":"estimators.py","file_url":"https://github.com/RakitinDen/pytorch-recursive-gumbel-max-trick/blob/HEAD/estimators.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"991472e4dbd1145d"}},{"code_sha256_prefix":"e624124dad9508c4","entry":"arb_mask_unused_values","repo":"RakitinDen/pytorch-recursive-gumbel-max-trick","repo_kind":"official","path":"arborescence/utils.py","file_url":"https://github.com/RakitinDen/pytorch-recursive-gumbel-max-trick/blob/HEAD/arborescence/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e624124dad9508c4"}},{"code_sha256_prefix":"d476d67388e3ef41","entry":"bin_tree_log_prob","repo":"RakitinDen/pytorch-recursive-gumbel-max-trick","repo_kind":"official","path":"binary_tree/functions.py","file_url":"https://github.com/RakitinDen/pytorch-recursive-gumbel-max-trick/blob/HEAD/binary_tree/functions.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d476d67388e3ef41"}},{"code_sha256_prefix":"ca8b8c5f518da92f","entry":"bin_tree_mask_unused_values","repo":"RakitinDen/pytorch-recursive-gumbel-max-trick","repo_kind":"official","path":"binary_tree/utils.py","file_url":"https://github.com/RakitinDen/pytorch-recursive-gumbel-max-trick/blob/HEAD/binary_tree/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ca8b8c5f518da92f"}},{"code_sha256_prefix":"7c52becd20da3056","entry":"build_tree","repo":"RakitinDen/pytorch-recursive-gumbel-max-trick","repo_kind":"official","path":"binary_tree/utils.py","file_url":"https://github.com/RakitinDen/pytorch-recursive-gumbel-max-trick/blob/HEAD/binary_tree/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7c52becd20da3056"}},{"code_sha256_prefix":"b6a193a098c58bcf","entry":"calc_trace_log_prob","repo":"RakitinDen/pytorch-recursive-gumbel-max-trick","repo_kind":"official","path":"arborescence/utils.py","file_url":"https://github.com/RakitinDen/pytorch-recursive-gumbel-max-trick/blob/HEAD/arborescence/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b6a193a098c58bcf"}},{"code_sha256_prefix":"ca07a89fcbf4abf8","entry":"expand_mask","repo":"RakitinDen/pytorch-recursive-gumbel-max-trick","repo_kind":"official","path":"arborescence/utils.py","file_url":"https://github.com/RakitinDen/pytorch-recursive-gumbel-max-trick/blob/HEAD/arborescence/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ca07a89fcbf4abf8"}},{"code_sha256_prefix":"f62fb7898212df4f","entry":"matching_log_prob","repo":"RakitinDen/pytorch-recursive-gumbel-max-trick","repo_kind":"official","path":"matching/functions.py","file_url":"https://github.com/RakitinDen/pytorch-recursive-gumbel-max-trick/blob/HEAD/matching/functions.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f62fb7898212df4f"}},{"code_sha256_prefix":"a48be52c9f177d0b","entry":"reattach_exp_to_new_logits","repo":"RakitinDen/pytorch-recursive-gumbel-max-trick","repo_kind":"official","path":"estimators.py","file_url":"https://github.com/RakitinDen/pytorch-recursive-gumbel-max-trick/blob/HEAD/estimators.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a48be52c9f177d0b"}},{"code_sha256_prefix":"e6843d775acab6e8","entry":"uniform_to_exp","repo":"RakitinDen/pytorch-recursive-gumbel-max-trick","repo_kind":"official","path":"estimators.py","file_url":"https://github.com/RakitinDen/pytorch-recursive-gumbel-max-trick/blob/HEAD/estimators.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e6843d775acab6e8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}