{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dynamic-neural-turing-machine-with-soft-and","title":"Dynamic Neural Turing Machine with Soft and Hard Addressing Schemes","arxiv_id":"1607.00036","date":"2016-06-30","proceeding":null,"authors":["Caglar Gulcehre","Sarath Chandar","Kyunghyun Cho","Yoshua Bengio"],"abstract":"We extend neural Turing machine (NTM) model into a dynamic neural Turing\nmachine (D-NTM) by introducing a trainable memory addressing scheme. This\naddressing scheme maintains for each memory cell two separate vectors, content\nand address vectors. This allows the D-NTM to learn a wide variety of\nlocation-based addressing strategies including both linear and nonlinear ones.\nWe implement the D-NTM with both continuous, differentiable and discrete,\nnon-differentiable read/write mechanisms. We investigate the mechanisms and\neffects of learning to read and write into a memory through experiments on\nFacebook bAbI tasks using both a feedforward and GRUcontroller. The D-NTM is\nevaluated on a set of Facebook bAbI tasks and shown to outperform NTM and LSTM\nbaselines. We have done extensive analysis of our model and different\nvariations of NTM on bAbI task. We also provide further experimental results on\nsequential pMNIST, Stanford Natural Language Inference, associative recall and\ncopy tasks.","url_abs":"http://arxiv.org/abs/1607.00036v2","url_pdf":"http://arxiv.org/pdf/1607.00036v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"natural-language-inference","task_name":"Natural Language Inference"},{"task_slug":"question-answering","task_name":"Question Answering"}],"methods":[{"method_slug":"content-based-attention","method_name":"Content-based Attention"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"location-based-attention","method_name":"Location-based Attention"},{"method_slug":"neural-turing-machine","method_name":"Neural Turing Machine"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/question-answering-on-babi","task":"Question Answering","dataset":"bAbi","model":"DMN+","rank_in_archive_order":5,"of":14,"metrics":{"Accuracy (trained on 10k)":"97.2%","Accuracy (trained on 1k)":"66.8%"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1607.00036","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}