{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/neural-estimation-of-the-rate-distortion","title":"Neural Estimation of the Rate-Distortion Function With Applications to Operational Source Coding","arxiv_id":"2204.01612","date":"2022-04-04","proceeding":null,"authors":["Eric Lei","Hamed Hassani","Shirin Saeedi Bidokhti"],"abstract":"A fundamental question in designing lossy data compression schemes is how well one can do in comparison with the rate-distortion function, which describes the known theoretical limits of lossy compression. Motivated by the empirical success of deep neural network (DNN) compressors on large, real-world data, we investigate methods to estimate the rate-distortion function on such data, which would allow comparison of DNN compressors with optimality. While one could use the empirical distribution of the data and apply the Blahut-Arimoto algorithm, this approach presents several computational challenges and inaccuracies when the datasets are large and high-dimensional, such as the case of modern image datasets. Instead, we re-formulate the rate-distortion objective, and solve the resulting functional optimization problem using neural networks. We apply the resulting rate-distortion estimator, called NERD, on popular image datasets, and provide evidence that NERD can accurately estimate the rate-distortion function. Using our estimate, we show that the rate-distortion achievable by DNN compressors are within several bits of the rate-distortion function for real-world datasets. Additionally, NERD provides access to the rate-distortion achieving channel, as well as samples from its output marginal. Therefore, using recent results in reverse channel coding, we describe how NERD can be used to construct an operational one-shot lossy compression scheme with guarantees on the achievable rate and distortion. Experimental results demonstrate competitive performance with DNN compressors.","url_abs":"https://arxiv.org/abs/2204.01612v2","url_pdf":"https://arxiv.org/pdf/2204.01612v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"neural-estimation-of-the-rate-distortion","repo_url":"https://github.com/leieric/nerd-rcc","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"data-compression","task_name":"Data Compression"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2204.01612","atlas_url":"https://app.syntology.ai/?focus=2204.01612","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2204.01612"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/leieric/nerd-rcc","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":6},"by_repo_kind":{"official":{"samples":6,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"59a1ffdf2299babd","entry":"est_dist","repo":"leieric/nerd-rcc","repo_kind":"official","path":"one_shot_PFRlagr.py","file_url":"https://github.com/leieric/nerd-rcc/blob/HEAD/one_shot_PFRlagr.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59a1ffdf2299babd"}},{"code_sha256_prefix":"601cb9e64ed9bd24","entry":"est_rate_zipf","repo":"leieric/nerd-rcc","repo_kind":"official","path":"one_shot_PFRlagr.py","file_url":"https://github.com/leieric/nerd-rcc/blob/HEAD/one_shot_PFRlagr.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"601cb9e64ed9bd24"}},{"code_sha256_prefix":"3d033b834aa52ef9","entry":"lam_obj","repo":"leieric/nerd-rcc","repo_kind":"official","path":"GaussianSource.py","file_url":"https://github.com/leieric/nerd-rcc/blob/HEAD/GaussianSource.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3d033b834aa52ef9"}},{"code_sha256_prefix":"d69e2a85f2aecb90","entry":"rd_gaussian","repo":"leieric/nerd-rcc","repo_kind":"official","path":"GaussianSource.py","file_url":"https://github.com/leieric/nerd-rcc/blob/HEAD/GaussianSource.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d69e2a85f2aecb90"}},{"code_sha256_prefix":"4544bfcafb08771c","entry":"rev_wf","repo":"leieric/nerd-rcc","repo_kind":"official","path":"GaussianSource.py","file_url":"https://github.com/leieric/nerd-rcc/blob/HEAD/GaussianSource.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4544bfcafb08771c"}},{"code_sha256_prefix":"995c1d63d01a457b","entry":"run_BlahutArimoto","repo":"leieric/nerd-rcc","repo_kind":"official","path":"Blahut_Arimoto/BlahutArimoto.py","file_url":"https://github.com/leieric/nerd-rcc/blob/HEAD/Blahut_Arimoto/BlahutArimoto.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"995c1d63d01a457b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}