{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-general-simd-based-approach-to-accelerating","title":"A General SIMD-based Approach to Accelerating Compression Algorithms","arxiv_id":"1502.01916","date":"2015-02-06","proceeding":null,"authors":["Wayne Xin Zhao","Xu-Dong Zhang","Daniel Lemire","Dongdong Shan","Jian-Yun Nie","Hongfei Yan","Ji-Rong Wen"],"abstract":"Compression algorithms are important for data oriented tasks, especially in\nthe era of Big Data. Modern processors equipped with powerful SIMD instruction\nsets, provide us an opportunity for achieving better compression performance.\nPrevious research has shown that SIMD-based optimizations can multiply decoding\nspeeds. Following these pioneering studies, we propose a general approach to\naccelerate compression algorithms. By instantiating the approach, we have\ndeveloped several novel integer compression algorithms, called Group-Simple,\nGroup-Scheme, Group-AFOR, and Group-PFD, and implemented their corresponding\nvectorized versions. We evaluate the proposed algorithms on two public TREC\ndatasets, a Wikipedia dataset and a Twitter dataset. With competitive\ncompression ratios and encoding speeds, our SIMD-based algorithms outperform\nstate-of-the-art non-vectorized algorithms with respect to decoding speeds.","url_abs":"http://arxiv.org/abs/1502.01916v1","url_pdf":"http://arxiv.org/pdf/1502.01916v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-general-simd-based-approach-to-accelerating","repo_url":"https://github.com/lemire/FastPFor","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}