{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/language-bootstrapping-learning-word-meanings","title":"Language Bootstrapping: Learning Word Meanings From Perception-Action Association","arxiv_id":"1711.09714","date":"2017-11-27","proceeding":null,"authors":["Giampiero Salvi","Luis Montesano","Alexandre Bernardino","José Santos-Victor"],"abstract":"We address the problem of bootstrapping language acquisition for an\nartificial system similarly to what is observed in experiments with human\ninfants. Our method works by associating meanings to words in manipulation\ntasks, as a robot interacts with objects and listens to verbal descriptions of\nthe interactions. The model is based on an affordance network, i.e., a mapping\nbetween robot actions, robot perceptions, and the perceived effects of these\nactions upon objects. We extend the affordance model to incorporate spoken\nwords, which allows us to ground the verbal symbols to the execution of actions\nand the perception of the environment. The model takes verbal descriptions of a\ntask as the input and uses temporal co-occurrence to create links between\nspeech utterances and the involved objects, actions, and effects. We show that\nthe robot is able form useful word-to-meaning associations, even without\nconsidering grammatical structure in the learning process and in the presence\nof recognition errors. These word-to-meaning associations are embedded in the\nrobot's own understanding of its actions. Thus, they can be directly used to\ninstruct the robot to perform tasks and also allow to incorporate context in\nthe speech recognition task. We believe that the encouraging results with our\napproach may afford robots with a capacity to acquire language descriptors in\ntheir operation's environment as well as to shed some light as to how this\nchallenging process develops with human infants.","url_abs":"http://arxiv.org/abs/1711.09714v1","url_pdf":"http://arxiv.org/pdf/1711.09714v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"language-bootstrapping-learning-word-meanings","repo_url":"https://github.com/giampierosalvi/AffordancesAndSpeech","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"language-acquisition","task_name":"Language Acquisition"},{"task_slug":"speech-recognition","task_name":"Speech Recognition"},{"task_slug":"speech-recognition-1","task_name":"speech-recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}