{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-neural-network-pushdown-automaton-model","title":"The Neural Network Pushdown Automaton: Model, Stack and Learning Simulations","arxiv_id":"1711.05738","date":"2017-11-15","proceeding":null,"authors":["G. Z. Sun","C. L. Giles","H. H. Chen","Y. C. Lee"],"abstract":"In order for neural networks to learn complex languages or grammars, they\nmust have sufficient computational power or resources to recognize or generate\nsuch languages. Though many approaches have been discussed, one ob- vious\napproach to enhancing the processing power of a recurrent neural network is to\ncouple it with an external stack memory - in effect creating a neural network\npushdown automata (NNPDA). This paper discusses in detail this NNPDA - its\nconstruction, how it can be trained and how useful symbolic information can be\nextracted from the trained network.\n  In order to couple the external stack to the neural network, an optimization\nmethod is developed which uses an error function that connects the learning of\nthe state automaton of the neural network to the learning of the operation of\nthe external stack. To minimize the error function using gradient descent\nlearning, an analog stack is designed such that the action and storage of\ninformation in the stack are continuous. One interpretation of a continuous\nstack is the probabilistic storage of and action on data. After training on\nsample strings of an unknown source grammar, a quantization procedure extracts\nfrom the analog stack and neural network a discrete pushdown automata (PDA).\nSimulations show that in learning deterministic context-free grammars - the\nbalanced parenthesis language, 1*n0*n, and the deterministic Palindrome - the\nextracted PDA is correct in the sense that it can correctly recognize unseen\nstrings of arbitrary length. In addition, the extracted PDAs can be shown to be\nidentical or equivalent to the PDAs of the source grammars which were used to\ngenerate the training strings.","url_abs":"http://arxiv.org/abs/1711.05738v1","url_pdf":"http://arxiv.org/pdf/1711.05738v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-neural-network-pushdown-automaton-model","repo_url":"https://github.com/eonu/arx","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"quantization","task_name":"Quantization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1711.05738","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}