{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-compression-scheme-for-radio-data-in-high","title":"A compression scheme for radio data in high performance computing","arxiv_id":"1503.00638","date":"2015-03-02","proceeding":null,"authors":["Kiyoshi Masui","Mandana Amiri","Liam Connor","Meiling Deng","Mateus Fandino","Carolin Höfer","Mark Halpern","David Hanna","Adam D. Hincks","Gary Hinshaw","Juan Mena Parra","Laura B. Newburgh","J. Richard Shaw","Keith Vanderlinde"],"abstract":"We present a procedure for efficiently compressing astronomical radio data for high performance applications. Integrated, post-correlation data are first passed through a nearly lossless rounding step which compares the precision of the data to a generalized and calibration-independent form of the radiometer equation. This allows the precision of the data to be reduced in a way that has an insignificant impact on the data. The newly developed Bitshuffle lossless compression algorithm is subsequently applied. When the algorithm is used in conjunction with the HDF5 library and data format, data produced by the CHIME Pathfinder telescope is compressed to 28% of its original size and decompression throughputs in excess of 1 GB/s are obtained on a single core.","url_abs":"http://arxiv.org/abs/1503.00638v3","url_pdf":"http://arxiv.org/pdf/1503.00638v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"a-compression-scheme-for-radio-data-in-high","repo_url":"https://github.com/kiyo-masui/bitshuffle","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"a-compression-scheme-for-radio-data-in-high","repo_url":"https://github.com/kernsuite-debian/bitshuffle","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}