{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/infobatch-lossless-training-speed-up-by","title":"InfoBatch: Lossless Training Speed Up by Unbiased Dynamic Data Pruning","arxiv_id":"2303.04947","date":"2023-03-08","proceeding":null,"authors":["Ziheng Qin","Kai Wang","Zangwei Zheng","Jianyang Gu","Xiangyu Peng","Zhaopan Xu","Daquan Zhou","Lei Shang","Baigui Sun","Xuansong Xie","Yang You"],"abstract":"Data pruning aims to obtain lossless performances with less overall cost. A common approach is to filter out samples that make less contribution to the training. This could lead to gradient expectation bias compared to the original data. To solve this problem, we propose \\textbf{InfoBatch}, a novel framework aiming to achieve lossless training acceleration by unbiased dynamic data pruning. Specifically, InfoBatch randomly prunes a portion of less informative samples based on the loss distribution and rescales the gradients of the remaining samples to approximate the original gradient. As a plug-and-play and architecture-agnostic framework, InfoBatch consistently obtains lossless training results on classification, semantic segmentation, vision pertaining, and instruction fine-tuning tasks. On CIFAR10/100, ImageNet-1K, and ADE20K, InfoBatch losslessly saves 40\\% overall cost. For pertaining MAE and diffusion model, InfoBatch can respectively save 24.8\\% and 27\\% cost. For LLaMA instruction fine-tuning, InfoBatch is also able to save 20\\% cost and is compatible with coreset selection methods. The code is publicly available at \\href{https://github.com/henryqin1997/InfoBatch}{github.com/NUS-HPC-AI-Lab/InfoBatch}.","url_abs":"https://arxiv.org/abs/2303.04947v2","url_pdf":"https://arxiv.org/pdf/2303.04947v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"infobatch-lossless-training-speed-up-by","repo_url":"https://github.com/nus-hpc-ai-lab/infobatch","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"lars","method_name":"LARS"},{"method_slug":"mae","method_name":"MAE"},{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2303.04947","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.04947"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nus-hpc-ai-lab/infobatch","reach":null}],"summary":{"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"09f4e10c77016d7e","entry":"concat_all_gather","repo":"nus-hpc-ai-lab/infobatch","repo_kind":"official","path":"infobatch/infobatch.py","file_url":"https://github.com/nus-hpc-ai-lab/infobatch/blob/HEAD/infobatch/infobatch.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"09f4e10c77016d7e"}},{"code_sha256_prefix":"d23c65f6e339df23","entry":"info_hack_indices","repo":"nus-hpc-ai-lab/infobatch","repo_kind":"official","path":"infobatch/infobatch.py","file_url":"https://github.com/nus-hpc-ai-lab/infobatch/blob/HEAD/infobatch/infobatch.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d23c65f6e339df23"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}