{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2607-06796","title":"Enhancing deep learning models for time series classification via knowledge distillation","arxiv_id":"2607.06796","date":"2026-07-07","proceeding":null,"authors":["Javidan Abdullayev","Maxime Devanne","Jonathan Weber","Germain Forestier"],"abstract":"Deep learning has achieved remarkable success in various domains including time series analysis, computer vision and natural language processing. However, high computational and memory demands of state-of-the-art architectures pose challenges for deployment in resource-limited environments. Knowledge Distillation (KD) addresses this by transferring knowledge from a large teacher model to a smaller, more efficient student model while maintaining competitive performance. In this work, we investigate the effectiveness of KD for Time Series Classification (TSC) across three architectures: the classical Fully Convolutional Network (FCN), the convolutional Inception model and the transformer-based ConvTran model. We evaluate our approach on UCR Archive, the largest benchmark repository of time series datasets, by modifying architectural components such as convolutional filters, Inception modules and attention heads across the three architectures. Our results consistently show that KD most effectively benefits student models of intermediate complexity across all three architectures, with the distilled FCN student reducing parameters by a factor of 38, the distilled Inception student achieving nearly the same performance as the teacher with 42% fewer parameters and the distilled ConvTran student with 2 attention heads showing the most significant improvement through distillation. To encourage further research and reproducibility, we provide our implementation at https://github.com/MSD-IRIMAS/KD-4-TSC.","url_abs":"https://arxiv.org/abs/2607.06796","url_pdf":"https://arxiv.org/pdf/2607.06796","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2607.06796","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2607.06796"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/MSD-IRIMAS/KD-4-TSC","reach":null}],"summary":{"ran":3,"ran_honours":1},"by_repo_kind":{"found_in_text":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"802e2c8e966af2b1","entry":"calculate_metrics","repo":"MSD-IRIMAS/KD-4-TSC","repo_kind":"found_in_text","path":"KD4TSC/utils.py","file_url":"https://github.com/MSD-IRIMAS/KD-4-TSC/blob/HEAD/KD4TSC/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"802e2c8e966af2b1"}},{"code_sha256_prefix":"f6b944f50d3f15ae","entry":"count_parameters","repo":"MSD-IRIMAS/KD-4-TSC","repo_kind":"found_in_text","path":"KD4TSC/models/convtran.py","file_url":"https://github.com/MSD-IRIMAS/KD-4-TSC/blob/HEAD/KD4TSC/models/convtran.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f6b944f50d3f15ae"}},{"code_sha256_prefix":"c5bd23970f6c07e2","entry":"evaluate_model","repo":"MSD-IRIMAS/KD-4-TSC","repo_kind":"found_in_text","path":"KD4TSC/utils.py","file_url":"https://github.com/MSD-IRIMAS/KD-4-TSC/blob/HEAD/KD4TSC/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c5bd23970f6c07e2"}},{"code_sha256_prefix":"8715c0778dfd9688","entry":"get_best_teacher","repo":"MSD-IRIMAS/KD-4-TSC","repo_kind":"found_in_text","path":"KD4TSC/utils.py","file_url":"https://github.com/MSD-IRIMAS/KD-4-TSC/blob/HEAD/KD4TSC/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8715c0778dfd9688"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}