{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/continual-learning-via-bit-level-information","title":"Continual Learning via Bit-Level Information Preserving","arxiv_id":"2105.04444","date":"2021-05-10","proceeding":"CVPR 2021 1","authors":["Yujun Shi","Li Yuan","Yunpeng Chen","Jiashi Feng"],"abstract":"Continual learning tackles the setting of learning different tasks sequentially. Despite the lots of previous solutions, most of them still suffer significant forgetting or expensive memory cost. In this work, targeted at these problems, we first study the continual learning process through the lens of information theory and observe that forgetting of a model stems from the loss of \\emph{information gain} on its parameters from the previous tasks when learning a new task. From this viewpoint, we then propose a novel continual learning approach called Bit-Level Information Preserving (BLIP) that preserves the information gain on model parameters through updating the parameters at the bit level, which can be conveniently implemented with parameter quantization. More specifically, BLIP first trains a neural network with weight quantization on the new incoming task and then estimates information gain on each parameter provided by the task data to determine the bits to be frozen to prevent forgetting. We conduct extensive experiments ranging from classification tasks to reinforcement learning tasks, and the results show that our method produces better or on par results comparing to previous state-of-the-arts. Indeed, BLIP achieves close to zero forgetting while only requiring constant memory overheads throughout continual learning.","url_abs":"https://arxiv.org/abs/2105.04444v4","url_pdf":"https://arxiv.org/pdf/2105.04444v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"continual-learning-via-bit-level-information","repo_url":"https://github.com/Yujun-Shi/BLIP","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"quantization","task_name":"Quantization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2105.04444","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2105.04444"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Yujun-Shi/BLIP","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":4},"by_repo_kind":{"official":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"02e5d161509b5aee","entry":"batch_conv","repo":"Yujun-Shi/BLIP","repo_kind":"official","path":"RL/rl_module/ppo_blip_utils.py","file_url":"https://github.com/Yujun-Shi/BLIP/blob/HEAD/RL/rl_module/ppo_blip_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"02e5d161509b5aee"}},{"code_sha256_prefix":"5c7a3f9d950dfbfe","entry":"evaluate","repo":"Yujun-Shi/BLIP","repo_kind":"official","path":"RL/rl_module/evaluation.py","file_url":"https://github.com/Yujun-Shi/BLIP/blob/HEAD/RL/rl_module/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5c7a3f9d950dfbfe"}},{"code_sha256_prefix":"59bcf578b74395ec","entry":"make_directories","repo":"Yujun-Shi/BLIP","repo_kind":"official","path":"ImageClassification/src/utils.py","file_url":"https://github.com/Yujun-Shi/BLIP/blob/HEAD/ImageClassification/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59bcf578b74395ec"}},{"code_sha256_prefix":"3b1fa928334d991d","entry":"print_log_acc_bwt","repo":"Yujun-Shi/BLIP","repo_kind":"official","path":"ImageClassification/src/utils.py","file_url":"https://github.com/Yujun-Shi/BLIP/blob/HEAD/ImageClassification/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b1fa928334d991d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}