{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/recurrent-models-for-situation-recognition","title":"Recurrent Models for Situation Recognition","arxiv_id":"1703.06233","date":"2017-03-18","proceeding":"ICCV 2017 10","authors":["Arun Mallya","Svetlana Lazebnik"],"abstract":"This work proposes Recurrent Neural Network (RNN) models to predict\nstructured 'image situations' -- actions and noun entities fulfilling semantic\nroles related to the action. In contrast to prior work relying on Conditional\nRandom Fields (CRFs), we use a specialized action prediction network followed\nby an RNN for noun prediction. Our system obtains state-of-the-art accuracy on\nthe challenging recent imSitu dataset, beating CRF-based models, including ones\ntrained with additional data. Further, we show that specialized features\nlearned from situation prediction can be transferred to the task of image\ncaptioning to more accurately describe human-object interactions.","url_abs":"http://arxiv.org/abs/1703.06233v2","url_pdf":"http://arxiv.org/pdf/1703.06233v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"grounded-situation-recognition","task_name":"Grounded Situation Recognition"},{"task_slug":"human-object-interaction-detection","task_name":"Human-Object Interaction Detection"},{"task_slug":"image-captioning","task_name":"Image Captioning"},{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"situation-recognition","task_name":"Situation Recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/grounded-situation-recognition-on-swig","task":"Grounded Situation Recognition","dataset":"SWiG","model":"RNN + Fusion","rank_in_archive_order":11,"of":13,"metrics":{"Top-1 Verb":"35.9","Top-1 Verb & Value":"27.45","Top-5 Verbs":"63.08","Top-5 Verbs & Value":"46.88"},"uses_additional_data":false},{"leaderboard":"/sota/situation-recognition-on-imsitu","task":"Situation Recognition","dataset":"imSitu","model":"RNN + Fusion","rank_in_archive_order":11,"of":13,"metrics":{"Top-1 Verb":"35.9","Top-1 Verb & Value":"27.45","Top-5 Verbs":"63.08","Top-5 Verbs & Value":"46.88"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1703.06233","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}