{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gradient-estimation-with-stochastic-softmax","title":"Gradient Estimation with Stochastic Softmax Tricks","arxiv_id":"2006.08063","date":"2020-06-15","proceeding":"NeurIPS 2020 12","authors":["Max B. Paulus","Dami Choi","Daniel Tarlow","Andreas Krause","Chris J. Maddison"],"abstract":"The Gumbel-Max trick is the basis of many relaxed gradient estimators. These estimators are easy to implement and low variance, but the goal of scaling them comprehensively to large combinatorial distributions is still outstanding. Working within the perturbation model framework, we introduce stochastic softmax tricks, which generalize the Gumbel-Softmax trick to combinatorial spaces. Our framework is a unified perspective on existing relaxed estimators for perturbation models, and it contains many novel relaxations. We design structured relaxations for subset selection, spanning trees, arborescences, and others. When compared to less structured baselines, we find that stochastic softmax tricks can be used to train latent variable models that perform better and discover more latent structure.","url_abs":"https://arxiv.org/abs/2006.08063v3","url_pdf":"https://arxiv.org/pdf/2006.08063v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gradient-estimation-with-stochastic-softmax","repo_url":"https://github.com/choidami/sst","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2006.08063","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2006.08063"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/choidami/sst","reach":null}],"summary":{"ran":1,"ran_draft_wrong":3},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"347bd2285e0766e5","entry":"SoftTopK","repo":"choidami/sst","repo_kind":"official","path":"core/topk.py","file_url":"https://github.com/choidami/sst/blob/HEAD/core/topk.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"347bd2285e0766e5"}},{"code_sha256_prefix":"93d8ed54bbbda1c2","entry":"softtopk_backward_np","repo":"choidami/sst","repo_kind":"official","path":"core/topk.py","file_url":"https://github.com/choidami/sst/blob/HEAD/core/topk.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"93d8ed54bbbda1c2"}},{"code_sha256_prefix":"80a039d224fcaa1c","entry":"softtopk_forward_np","repo":"choidami/sst","repo_kind":"official","path":"core/topk.py","file_url":"https://github.com/choidami/sst/blob/HEAD/core/topk.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"80a039d224fcaa1c"}},{"code_sha256_prefix":"abbf3df2f69439d7","entry":"softtopk_np","repo":"choidami/sst","repo_kind":"official","path":"core/topk.py","file_url":"https://github.com/choidami/sst/blob/HEAD/core/topk.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"abbf3df2f69439d7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}