{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/training-for-the-future-a-simple-gradient","title":"Training for the Future: A Simple Gradient Interpolation Loss to Generalize Along Time","arxiv_id":"2108.06721","date":"2021-08-15","proceeding":"NeurIPS 2021 12","authors":["Anshul Nasery","Soumyadeep Thakur","Vihari Piratla","Abir De","Sunita Sarawagi"],"abstract":"In several real world applications, machine learning models are deployed to make predictions on data whose distribution changes gradually along time, leading to a drift between the train and test distributions. Such models are often re-trained on new data periodically, and they hence need to generalize to data not too far into the future. In this context, there is much prior work on enhancing temporal generalization, e.g. continuous transportation of past data, kernel smoothed time-sensitive parameters and more recently, adversarial learning of time-invariant features. However, these methods share several limitations, e.g, poor scalability, training instability, and dependence on unlabeled data from the future. Responding to the above limitations, we propose a simple method that starts with a model with time-sensitive parameters but regularizes its temporal complexity using a Gradient Interpolation (GI) loss. GI allows the decision boundary to change along time and can still prevent overfitting to the limited training time snapshots by allowing task-specific control over changes along time. We compare our method to existing baselines on multiple real-world datasets, which show that GI outperforms more complicated generative and adversarial approaches on the one hand, and simpler gradient regularization methods on the other.","url_abs":"https://arxiv.org/abs/2108.06721v2","url_pdf":"https://arxiv.org/pdf/2108.06721v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"training-for-the-future-a-simple-gradient","repo_url":"https://github.com/anshuln/training-for-the-future","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2108.06721","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2108.06721"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/anshuln/training-for-the-future","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":7},"by_repo_kind":{"official":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"916f3b5d7666cb5a","entry":"load_data","repo":"anshuln/training-for-the-future","repo_kind":"official","path":"src/CDOT/ot_main.py","file_url":"https://github.com/anshuln/training-for-the-future/blob/HEAD/src/CDOT/ot_main.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"916f3b5d7666cb5a"}},{"code_sha256_prefix":"1272a3ba576cd814","entry":"masked_cross_entropy","repo":"anshuln/training-for-the-future","repo_kind":"official","path":"src/CIDA/utils.py","file_url":"https://github.com/anshuln/training-for-the-future/blob/HEAD/src/CIDA/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1272a3ba576cd814"}},{"code_sha256_prefix":"fb4e99638972135b","entry":"masked_mse","repo":"anshuln/training-for-the-future","repo_kind":"official","path":"src/CIDA/utils.py","file_url":"https://github.com/anshuln/training-for-the-future/blob/HEAD/src/CIDA/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fb4e99638972135b"}},{"code_sha256_prefix":"f8bfab7b7811c4b5","entry":"naive_transport","repo":"anshuln/training-for-the-future","repo_kind":"official","path":"src/CDOT/transport.py","file_url":"https://github.com/anshuln/training-for-the-future/blob/HEAD/src/CDOT/transport.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f8bfab7b7811c4b5"}},{"code_sha256_prefix":"078fd72e0d08607d","entry":"read_data","repo":"anshuln/training-for-the-future","repo_kind":"official","path":"src/CIDA/data_loader.py","file_url":"https://github.com/anshuln/training-for-the-future/blob/HEAD/src/CIDA/data_loader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"078fd72e0d08607d"}},{"code_sha256_prefix":"3e3f8e045455f655","entry":"read_pickle","repo":"anshuln/training-for-the-future","repo_kind":"official","path":"src/CIDA/utils.py","file_url":"https://github.com/anshuln/training-for-the-future/blob/HEAD/src/CIDA/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3e3f8e045455f655"}},{"code_sha256_prefix":"0c917287552e370b","entry":"train_classifier","repo":"anshuln/training-for-the-future","repo_kind":"official","path":"src/CDOT/ot_main.py","file_url":"https://github.com/anshuln/training-for-the-future/blob/HEAD/src/CDOT/ot_main.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0c917287552e370b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}