{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multi-granularity-reasoning-for-social","title":"Multi-Granularity Reasoning for Social Relation Recognition from Images","arxiv_id":"1901.03067","date":"2019-01-10","proceeding":null,"authors":["Meng Zhang","Xinchen Liu","Wu Liu","Anfu Zhou","Huadong Ma","Tao Mei"],"abstract":"Discovering social relations in images can make machines better interpret the\nbehavior of human beings. However, automatically recognizing social relations\nin images is a challenging task due to the significant gap between the domains\nof visual content and social relation. Existing studies separately process\nvarious features such as faces expressions, body appearance, and contextual\nobjects, thus they cannot comprehensively capture the multi-granularity\nsemantics, such as scenes, regional cues of persons, and interactions among\npersons and objects. To bridge the domain gap, we propose a Multi-Granularity\nReasoning framework for social relation recognition from images. The global\nknowledge and mid-level details are learned from the whole scene and the\nregions of persons and objects, respectively. Most importantly, we explore the\nfine-granularity pose keypoints of persons to discover the interactions among\npersons and objects. Specifically, the pose-guided Person-Object Graph and\nPerson-Pose Graph are proposed to model the actions from persons to object and\nthe interactions between paired persons, respectively. Based on the graphs,\nsocial relation reasoning is performed by graph convolutional networks.\nFinally, the global features and reasoned knowledge are integrated as a\ncomprehensive representation for social relation recognition. Extensive\nexperiments on two public datasets show the effectiveness of the proposed\nframework.","url_abs":"http://arxiv.org/abs/1901.03067v1","url_pdf":"http://arxiv.org/pdf/1901.03067v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":null,"task_name":"Relation"},{"task_slug":"visual-social-relationship-recognition","task_name":"Visual Social Relationship Recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/visual-social-relationship-recognition-on","task":"Visual Social Relationship Recognition","dataset":"PISC","model":"MGR","rank_in_archive_order":3,"of":5,"metrics":{"mAP":"70.0"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1901.03067","atlas_url":"https://app.syntology.ai/?focus=1901.03067","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}