{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-to-remember-more-with-less","title":"Learning to Remember More with Less Memorization","arxiv_id":"1901.01347","date":"2019-01-05","proceeding":"ICLR 2019 5","authors":["Hung Le","Truyen Tran","Svetha Venkatesh"],"abstract":"Memory-augmented neural networks consisting of a neural controller and an\nexternal memory have shown potentials in long-term sequential learning. Current\nRAM-like memory models maintain memory accessing every timesteps, thus they do\nnot effectively leverage the short-term memory held in the controller. We\nhypothesize that this scheme of writing is suboptimal in memory utilization and\nintroduces redundant computation. To validate our hypothesis, we derive a\ntheoretical bound on the amount of information stored in a RAM-like system and\nformulate an optimization problem that maximizes the bound. The proposed\nsolution dubbed Uniform Writing is proved to be optimal under the assumption of\nequal timestep contributions. To relax this assumption, we introduce\nmodifications to the original solution, resulting in a solution termed Cached\nUniform Writing. This method aims to balance between maximizing memorization\nand forgetting via overwriting mechanisms. Through an extensive set of\nexperiments, we empirically demonstrate the advantages of our solutions over\nother recurrent architectures, claiming the state-of-the-arts in various\nsequential modeling tasks.","url_abs":"http://arxiv.org/abs/1901.01347v2","url_pdf":"http://arxiv.org/pdf/1901.01347v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-to-remember-more-with-less","repo_url":"https://github.com/thaihungle/UW-DNC","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"memorization","task_name":"Memorization"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"},{"task_slug":"sequential-image-classification","task_name":"Sequential Image Classification"},{"task_slug":"text-classification","task_name":"Text Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/sentiment-analysis-on-yelp-binary","task":"Sentiment Analysis","dataset":"Yelp Binary classification","model":"DNC+CUW","rank_in_archive_order":13,"of":20,"metrics":{"Error":"3.60"},"uses_additional_data":false},{"leaderboard":"/sota/sentiment-analysis-on-yelp-fine-grained","task":"Sentiment Analysis","dataset":"Yelp Fine-grained classification","model":"DNC+CUW","rank_in_archive_order":12,"of":17,"metrics":{"Error":"34.40"},"uses_additional_data":false},{"leaderboard":"/sota/sequential-image-classification-on-sequential","task":"Sequential Image Classification","dataset":"Sequential MNIST","model":"DNC+CUW","rank_in_archive_order":20,"of":30,"metrics":{"Permuted Accuracy":"96.3%","Unpermuted Accuracy":"99.1%"},"uses_additional_data":false},{"leaderboard":"/sota/text-classification-on-ag-news","task":"Text Classification","dataset":"AG News","model":"DNC+CUW","rank_in_archive_order":6,"of":24,"metrics":{"Error":"6.10"},"uses_additional_data":false},{"leaderboard":"/sota/text-classification-on-yahoo-answers","task":"Text Classification","dataset":"Yahoo! Answers","model":"DNC+CUW","rank_in_archive_order":5,"of":10,"metrics":{"Accuracy":"74.30"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1901.01347","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1901.01347"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thaihungle/UW-DNC","reach":null}],"summary":{"ran_fixture":1,"ran_draft_wrong":1,"ran_honours":1},"by_repo_kind":{"listed":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3786f12279ce538c","entry":"exact_acc","repo":"thaihungle/UW-DNC","repo_kind":"listed","path":"synthetic_task.py","file_url":"https://github.com/thaihungle/UW-DNC/blob/HEAD/synthetic_task.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3786f12279ce538c"}},{"code_sha256_prefix":"c03957231341007b","entry":"load","repo":"thaihungle/UW-DNC","repo_kind":"listed","path":"synthetic_task.py","file_url":"https://github.com/thaihungle/UW-DNC/blob/HEAD/synthetic_task.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c03957231341007b"}},{"code_sha256_prefix":"b0584c5b4ca70059","entry":"onehot","repo":"thaihungle/UW-DNC","repo_kind":"listed","path":"synthetic_task.py","file_url":"https://github.com/thaihungle/UW-DNC/blob/HEAD/synthetic_task.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b0584c5b4ca70059"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}