{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generating-images-with-sparse-representations","title":"Generating Images with Sparse Representations","arxiv_id":"2103.03841","date":"2021-03-05","proceeding":null,"authors":["Charlie Nash","Jacob Menick","Sander Dieleman","Peter W. Battaglia"],"abstract":"The high dimensionality of images presents architecture and sampling-efficiency challenges for likelihood-based generative models. Previous approaches such as VQ-VAE use deep autoencoders to obtain compact representations, which are more practical as inputs for likelihood-based models. We present an alternative approach, inspired by common image compression methods like JPEG, and convert images to quantized discrete cosine transform (DCT) blocks, which are represented sparsely as a sequence of DCT channel, spatial location, and DCT coefficient triples. We propose a Transformer-based autoregressive architecture, which is trained to sequentially predict the conditional distribution of the next element in such sequences, and which scales effectively to high resolution images. On a range of image datasets, we demonstrate that our approach can generate high quality, diverse images, with sample metric scores competitive with state of the art methods. We additionally show that simple modifications to our method yield effective image colorization and super-resolution models.","url_abs":"https://arxiv.org/abs/2103.03841v1","url_pdf":"https://arxiv.org/pdf/2103.03841v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generating-images-with-sparse-representations","repo_url":"https://github.com/MindCode-4/code-11/tree/main/DCTransformer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"generating-images-with-sparse-representations","repo_url":"https://github.com/benjs/DCTransformer-PyTorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"colorization","task_name":"Colorization"},{"task_slug":"image-colorization","task_name":"Image Colorization"},{"task_slug":"image-compression","task_name":"Image Compression"},{"task_slug":"super-resolution","task_name":"Super-Resolution"}],"methods":[{"method_slug":"colorization","method_name":"Colorization"},{"method_slug":"discrete-cosine-transform","method_name":"Discrete Cosine Transform"},{"method_slug":"vq-vae","method_name":"VQ-VAE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2103.03841","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2103.03841"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/benjs/DCTransformer-PyTorch","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindCode-4/code-11/tree/main/DCTransformer","reach":null}],"summary":{"ran_honours":2},"by_repo_kind":{"listed":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bf2e115e75c42dfc","entry":"Q_luma","repo":"benjs/DCTransformer-PyTorch","repo_kind":"listed","path":"dctransformer/transforms.py","file_url":"https://github.com/benjs/DCTransformer-PyTorch/blob/HEAD/dctransformer/transforms.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bf2e115e75c42dfc"}},{"code_sha256_prefix":"b766941d68636838","entry":"zigzag","repo":"benjs/DCTransformer-PyTorch","repo_kind":"listed","path":"dctransformer/transforms.py","file_url":"https://github.com/benjs/DCTransformer-PyTorch/blob/HEAD/dctransformer/transforms.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b766941d68636838"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}