{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-concrete-distribution-a-continuous","title":"The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables","arxiv_id":"1611.00712","date":"2016-11-02","proceeding":null,"authors":["Chris J. Maddison","andriy mnih","Yee Whye Teh"],"abstract":"The reparameterization trick enables optimizing large scale stochastic\ncomputation graphs via gradient descent. The essence of the trick is to\nrefactor each stochastic node into a differentiable function of its parameters\nand a random variable with fixed distribution. After refactoring, the gradients\nof the loss propagated by the chain rule through the graph are low variance\nunbiased estimators of the gradients of the expected loss. While many\ncontinuous random variables have such reparameterizations, discrete random\nvariables lack useful reparameterizations due to the discontinuous nature of\ndiscrete states. In this work we introduce Concrete random\nvariables---continuous relaxations of discrete random variables. The Concrete\ndistribution is a new family of distributions with closed form densities and a\nsimple reparameterization. Whenever a discrete stochastic node of a computation\ngraph can be refactored into a one-hot bit representation that is treated\ncontinuously, Concrete stochastic nodes can be used with automatic\ndifferentiation to produce low-variance biased gradients of objectives\n(including objectives that depend on the log-probability of latent stochastic\nnodes) on the corresponding discrete graph. We demonstrate the effectiveness of\nConcrete relaxations on density estimation and structured prediction tasks\nusing neural networks.","url_abs":"http://arxiv.org/abs/1611.00712v3","url_pdf":"http://arxiv.org/pdf/1611.00712v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-concrete-distribution-a-continuous","repo_url":"https://github.com/aurelio-amerio/concretedropout","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"the-concrete-distribution-a-continuous","repo_url":"https://github.com/kampta/pytorch-distributions","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"the-concrete-distribution-a-continuous","repo_url":"https://github.com/leequant761/Gumbel-SSVAE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"the-concrete-distribution-a-continuous","repo_url":"https://github.com/tensorflow/models","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"the-concrete-distribution-a-continuous","repo_url":"https://github.com/tensorflow/models/tree/master/research/rebar","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"density-estimation","task_name":"Density Estimation"},{"task_slug":"structured-prediction","task_name":"Structured Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1611.00712","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1611.00712"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tensorflow/models","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/leequant761/Gumbel-SSVAE","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tensorflow/models/tree/master/research/rebar","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kampta/pytorch-distributions","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/aurelio-amerio/concretedropout","reach":null}],"summary":{"ran":3,"unverified":2},"by_repo_kind":{"listed":{"samples":5,"ran":3,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"cdc2a06469b6e130","entry":"Classifier","repo":"leequant761/Gumbel-SSVAE","repo_kind":"listed","path":"models.py","file_url":"https://github.com/leequant761/Gumbel-SSVAE/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cdc2a06469b6e130"}},{"code_sha256_prefix":"59cbe9a309055883","entry":"Decoder","repo":"leequant761/Gumbel-SSVAE","repo_kind":"listed","path":"models.py","file_url":"https://github.com/leequant761/Gumbel-SSVAE/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"59cbe9a309055883"}},{"code_sha256_prefix":"366f5ac5d303dbc7","entry":"Encoder","repo":"leequant761/Gumbel-SSVAE","repo_kind":"listed","path":"models.py","file_url":"https://github.com/leequant761/Gumbel-SSVAE/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"366f5ac5d303dbc7"}},{"code_sha256_prefix":"f80f227eb698b316","entry":"ConcreteDropout","repo":"aurelio-amerio/concretedropout","repo_kind":"listed","path":"src/ConcreteDropout/tensorflow.py","file_url":"https://github.com/aurelio-amerio/concretedropout/blob/HEAD/src/ConcreteDropout/tensorflow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f80f227eb698b316"}},{"code_sha256_prefix":"d68fa17739948be8","entry":"Model","repo":"leequant761/Gumbel-SSVAE","repo_kind":"listed","path":"models.py","file_url":"https://github.com/leequant761/Gumbel-SSVAE/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d68fa17739948be8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}