{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/pixelsnail-an-improved-autoregressive","title":"PixelSNAIL: An Improved Autoregressive Generative Model","arxiv_id":"1712.09763","date":"2017-12-28","proceeding":"ICML 2018 7","authors":["Xi Chen","Nikhil Mishra","Mostafa Rohaninejad","Pieter Abbeel"],"abstract":"Autoregressive generative models consistently achieve the best results in\ndensity estimation tasks involving high dimensional data, such as images or\naudio. They pose density estimation as a sequence modeling task, where a\nrecurrent neural network (RNN) models the conditional distribution over the\nnext element conditioned on all previous elements. In this paradigm, the\nbottleneck is the extent to which the RNN can model long-range dependencies,\nand the most successful approaches rely on causal convolutions, which offer\nbetter access to earlier parts of the sequence than conventional RNNs. Taking\ninspiration from recent work in meta reinforcement learning, where dealing with\nlong-range dependencies is also essential, we introduce a new generative model\narchitecture that combines causal convolutions with self attention. In this\nnote, we describe the resulting model and present state-of-the-art\nlog-likelihood results on CIFAR-10 (2.85 bits per dim) and $32 \\times 32$\nImageNet (3.80 bits per dim). Our implementation is available at\nhttps://github.com/neocxi/pixelsnail-public","url_abs":"http://arxiv.org/abs/1712.09763v1","url_pdf":"http://arxiv.org/pdf/1712.09763v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"pixelsnail-an-improved-autoregressive","repo_url":"https://github.com/neocxi/pixelsnail-public","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"pixelsnail-an-improved-autoregressive","repo_url":"https://github.com/kamenbliznashki/pixel_models","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"pixelsnail-an-improved-autoregressive","repo_url":"https://github.com/mattiasxu/VQVAE-2","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"pixelsnail-an-improved-autoregressive","repo_url":"https://github.com/mattiasxu/Video-VQVAE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"pixelsnail-an-improved-autoregressive","repo_url":"https://github.com/EugenHotaj/pytorch-generative/blob/master/pytorch_generative/models/autoregressive/pixel_snail.py","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"pixelsnail-an-improved-autoregressive","repo_url":"https://github.com/sarus-tech/tf2-published-models/tree/master/pixelsnail","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"density-estimation","task_name":"Density Estimation"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"meta-reinforcement-learning","task_name":"Meta Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"model","task_name":"model"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1712.09763","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1712.09763"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mattiasxu/Video-VQVAE","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kamenbliznashki/pixel_models","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mattiasxu/VQVAE-2","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/EugenHotaj/pytorch-generative/blob/master/pytorch_generative/models/autoregressive/pixel_snail.py","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sarus-tech/tf2-published-models/tree/master/pixelsnail","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/neocxi/pixelsnail-public","reach":{"status":"unanswered"}}],"summary":{"ran_honours":1,"ran_draft_wrong":4,"ran_fixture":1,"unverified":1},"by_repo_kind":{"listed":{"samples":7,"ran":6,"repositories":3}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"3b63dd544bcda245","entry":"causal_mask","repo":"mattiasxu/Video-VQVAE","repo_kind":"listed","path":"models/pixelsnail.py","file_url":"https://github.com/mattiasxu/Video-VQVAE/blob/HEAD/models/pixelsnail.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b63dd544bcda245"}},{"code_sha256_prefix":"1bc8478089c8a7c9","entry":"concat_elu","repo":"kamenbliznashki/pixel_models","repo_kind":"listed","path":"pixelsnail.py","file_url":"https://github.com/kamenbliznashki/pixel_models/blob/HEAD/pixelsnail.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1bc8478089c8a7c9"}},{"code_sha256_prefix":"a65cc793ef279a03","entry":"down_shift","repo":"kamenbliznashki/pixel_models","repo_kind":"listed","path":"pixelsnail.py","file_url":"https://github.com/kamenbliznashki/pixel_models/blob/HEAD/pixelsnail.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a65cc793ef279a03"}},{"code_sha256_prefix":"4b9f2b42b68582a2","entry":"elu_conv_elu","repo":"mattiasxu/Video-VQVAE","repo_kind":"listed","path":"models/pixelsnail.py","file_url":"https://github.com/mattiasxu/Video-VQVAE/blob/HEAD/models/pixelsnail.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4b9f2b42b68582a2"}},{"code_sha256_prefix":"600797d71ef88fc2","entry":"positional_encoding","repo":"mattiasxu/Video-VQVAE","repo_kind":"listed","path":"models/pixelsnail.py","file_url":"https://github.com/mattiasxu/Video-VQVAE/blob/HEAD/models/pixelsnail.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"600797d71ef88fc2"}},{"code_sha256_prefix":"1074bb941737a16e","entry":"right_shift","repo":"kamenbliznashki/pixel_models","repo_kind":"listed","path":"pixelsnail.py","file_url":"https://github.com/kamenbliznashki/pixel_models/blob/HEAD/pixelsnail.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1074bb941737a16e"}},{"code_sha256_prefix":"2319e2b227c34263","entry":"reproduce","repo":"EugenHotaj/pytorch-generative","repo_kind":"listed","path":"pytorch_generative/models/autoregressive/pixel_snail.py","file_url":"https://github.com/EugenHotaj/pytorch-generative/blob/HEAD/pytorch_generative/models/autoregressive/pixel_snail.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2319e2b227c34263"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}