{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deepzip-lossless-data-compression-using","title":"DeepZip: Lossless Data Compression using Recurrent Neural Networks","arxiv_id":"1811.08162","date":"2018-11-20","proceeding":null,"authors":["Mohit Goyal","Kedar Tatwawadi","Shubham Chandak","Idoia Ochoa"],"abstract":"Sequential data is being generated at an unprecedented pace in various forms,\nincluding text and genomic data. This creates the need for efficient\ncompression mechanisms to enable better storage, transmission and processing of\nsuch data. To solve this problem, many of the existing compressors attempt to\nlearn models for the data and perform prediction-based compression. Since\nneural networks are known as universal function approximators with the\ncapability to learn arbitrarily complex mappings, and in practice show\nexcellent performance in prediction tasks, we explore and devise methods to\ncompress sequential data using neural network predictors. We combine recurrent\nneural network predictors with an arithmetic coder and losslessly compress a\nvariety of synthetic, text and genomic datasets. The proposed compressor\noutperforms Gzip on the real datasets and achieves near-optimal compression for\nthe synthetic datasets. The results also help understand why and where neural\nnetworks are good alternatives for traditional finite context models","url_abs":"http://arxiv.org/abs/1811.08162v1","url_pdf":"http://arxiv.org/pdf/1811.08162v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deepzip-lossless-data-compression-using","repo_url":"https://github.com/mohit1997/DeepZip","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"data-compression","task_name":"Data Compression"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1811.08162","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1811.08162"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mohit1997/DeepZip","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":4},"by_repo_kind":{"official":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8157c6c0bc45b3e3","entry":"create_data","repo":"mohit1997/DeepZip","repo_kind":"official","path":"src/decompressor.py","file_url":"https://github.com/mohit1997/DeepZip/blob/HEAD/src/decompressor.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8157c6c0bc45b3e3"}},{"code_sha256_prefix":"2f674fc0c0d07370","entry":"generate_single_output_data","repo":"mohit1997/DeepZip","repo_kind":"official","path":"src/trainer.py","file_url":"https://github.com/mohit1997/DeepZip/blob/HEAD/src/trainer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2f674fc0c0d07370"}},{"code_sha256_prefix":"d10e845597405b7e","entry":"loss_fn","repo":"mohit1997/DeepZip","repo_kind":"official","path":"src/trainer.py","file_url":"https://github.com/mohit1997/DeepZip/blob/HEAD/src/trainer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d10e845597405b7e"}},{"code_sha256_prefix":"3b5c6179b8c17f7f","entry":"strided_app","repo":"mohit1997/DeepZip","repo_kind":"official","path":"src/compressor.py","file_url":"https://github.com/mohit1997/DeepZip/blob/HEAD/src/compressor.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b5c6179b8c17f7f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}