{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-human-activities-and-object","title":"Learning Human Activities and Object Affordances from RGB-D Videos","arxiv_id":"1210.1207","date":"2012-10-04","proceeding":null,"authors":["Hema Swetha Koppula","Rudhir Gupta","Ashutosh Saxena"],"abstract":"Understanding human activities and object affordances are two very important\nskills, especially for personal robots which operate in human environments. In\nthis work, we consider the problem of extracting a descriptive labeling of the\nsequence of sub-activities being performed by a human, and more importantly, of\ntheir interactions with the objects in the form of associated affordances.\nGiven a RGB-D video, we jointly model the human activities and object\naffordances as a Markov random field where the nodes represent objects and\nsub-activities, and the edges represent the relationships between object\naffordances, their relations with sub-activities, and their evolution over\ntime. We formulate the learning problem using a structural support vector\nmachine (SSVM) approach, where labelings over various alternate temporal\nsegmentations are considered as latent variables. We tested our method on a\nchallenging dataset comprising 120 activity videos collected from 4 subjects,\nand obtained an accuracy of 79.4% for affordance, 63.4% for sub-activity and\n75.0% for high-level activity labeling. We then demonstrate the use of such\ndescriptive labeling in performing assistive tasks by a PR2 robot.","url_abs":"http://arxiv.org/abs/1210.1207v2","url_pdf":"http://arxiv.org/pdf/1210.1207v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"descriptive","task_name":"Descriptive"},{"task_slug":"object","task_name":"Object"},{"task_slug":"skeleton-based-action-recognition","task_name":"Skeleton Based Action Recognition"}],"methods":[],"datasets_introduced":[{"slug":"cad-120","name":"CAD-120","full_name":""}],"methods_introduced":[],"results":[{"leaderboard":"/sota/skeleton-based-action-recognition-on-cad-120","task":"Skeleton Based Action Recognition","dataset":"CAD-120","model":"KGS","rank_in_archive_order":3,"of":8,"metrics":{"Accuracy":"86.0%"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1210.1207","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}