{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/incremental-sequence-learning","title":"Incremental Sequence Learning","arxiv_id":"1611.03068","date":"2016-11-09","proceeding":null,"authors":["Edwin D. de Jong"],"abstract":"Deep learning research over the past years has shown that by increasing the\nscope or difficulty of the learning problem over time, increasingly complex\nlearning problems can be addressed. We study incremental learning in the\ncontext of sequence learning, using generative RNNs in the form of multi-layer\nrecurrent Mixture Density Networks. While the potential of incremental or\ncurriculum learning to enhance learning is known, indiscriminate application of\nthe principle does not necessarily lead to improvement, and it is essential\ntherefore to know which forms of incremental or curriculum learning have a\npositive effect. This research contributes to that aim by comparing three\ninstantiations of incremental or curriculum learning.\n  We introduce Incremental Sequence Learning, a simple incremental approach to\nsequence learning. Incremental Sequence Learning starts out by using only the\nfirst few steps of each sequence as training data. Each time a performance\ncriterion has been reached, the length of the parts of the sequences used for\ntraining is increased.\n  We introduce and make available a novel sequence learning task and data set:\npredicting and classifying MNIST pen stroke sequences. We find that Incremental\nSequence Learning greatly speeds up sequence learning and reaches the best test\nperformance level of regular sequence learning 20 times faster, reduces the\ntest error by 74%, and in general performs more robustly; it displays lower\nvariance and achieves sustained progress after all three comparison methods\nhave stopped improving. The other instantiations of curriculum learning do not\nresult in any noticeable improvement. A trained sequence prediction model is\nalso used in transfer learning to the task of sequence classification, where it\nis found that transfer learning realizes improved classification performance\ncompared to methods that learn to classify from scratch.","url_abs":"http://arxiv.org/abs/1611.03068v2","url_pdf":"http://arxiv.org/pdf/1611.03068v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"incremental-sequence-learning","repo_url":"https://github.com/edwin-de-jong/incremental-sequence-learning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"incremental-learning","task_name":"Incremental Learning"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1611.03068","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1611.03068"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/edwin-de-jong/incremental-sequence-learning","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7bb85eb640efb52d","entry":"get_pi_idx","repo":"edwin-de-jong/incremental-sequence-learning","repo_kind":"official","path":"model.py","file_url":"https://github.com/edwin-de-jong/incremental-sequence-learning/blob/HEAD/model.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7bb85eb640efb52d"}},{"code_sha256_prefix":"bf183031cbfcd737","entry":"get_bounds","repo":"edwin-de-jong/incremental-sequence-learning","repo_kind":"official","path":"utils.py","file_url":"https://github.com/edwin-de-jong/incremental-sequence-learning/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bf183031cbfcd737"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}