{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/vision-based-deep-execution-monitoring","title":"Vision-based deep execution monitoring","arxiv_id":"1709.10507","date":"2017-09-29","proceeding":null,"authors":["Francesco Puja","Simone Grazioso","Antonio Tammaro","Valsmis Ntouskos","Marta Sanzari","Fiora Pirri"],"abstract":"Execution monitor of high-level robot actions can be effectively improved by\nvisual monitoring the state of the world in terms of preconditions and\npostconditions that hold before and after the execution of an action.\nFurthermore a policy for searching where to look at, either for verifying the\nrelations that specify the pre and postconditions or to refocus in case of a\nfailure, can tremendously improve the robot execution in an uncharted\nenvironment. It is now possible to strongly rely on visual perception in order\nto make the assumption that the environment is observable, by the amazing\nresults of deep learning. In this work we present visual execution monitoring\nfor a robot executing tasks in an uncharted Lab environment. The execution\nmonitor interacts with the environment via a visual stream that uses two DCNN\nfor recognizing the objects the robot has to deal with and manipulate, and a\nnon-parametric Bayes estimation to discover the relations out of the DCNN\nfeatures. To recover from lack of focus and failures due to missed objects we\nresort to visual search policies via deep reinforcement learning.","url_abs":"http://arxiv.org/abs/1709.10507v1","url_pdf":"http://arxiv.org/pdf/1709.10507v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"vision-based-deep-execution-monitoring","repo_url":"https://github.com/miyosuda/async_deep_reinforce","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}