{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/pmi-sampler-patch-similarity-guided-frame","title":"PMI Sampler: Patch Similarity Guided Frame Selection for Aerial Action Recognition","arxiv_id":"2304.06866","date":"2023-04-14","proceeding":null,"authors":["Ruiqi Xian","Xijun Wang","Divya Kothandaraman","Dinesh Manocha"],"abstract":"We present a new algorithm for selection of informative frames in video action recognition. Our approach is designed for aerial videos captured using a moving camera where human actors occupy a small spatial resolution of video frames. Our algorithm utilizes the motion bias within aerial videos, which enables the selection of motion-salient frames. We introduce the concept of patch mutual information (PMI) score to quantify the motion bias between adjacent frames, by measuring the similarity of patches. We use this score to assess the amount of discriminative motion information contained in one frame relative to another. We present an adaptive frame selection strategy using shifted leaky ReLu and cumulative distribution function, which ensures that the sampled frames comprehensively cover all the essential segments with high motion salience. Our approach can be integrated with any action recognition model to enhance its accuracy. In practice, our method achieves a relative improvement of 2.2 - 13.8% in top-1 accuracy on UAV-Human, 6.8% on NEC Drone, and 9.0% on Diving48 datasets.","url_abs":"https://arxiv.org/abs/2304.06866v2","url_pdf":"https://arxiv.org/pdf/2304.06866v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"pmi-sampler-patch-similarity-guided-frame","repo_url":"https://github.com/ricky-xian/pmi-sampler","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"action-recognition","task_name":"Temporal Action Localization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/action-recognition-on-diving-48","task":"Action Recognition","dataset":"Diving-48","model":"PMI Sampler","rank_in_archive_order":14,"of":18,"metrics":{"Accuracy":"81.3"},"uses_additional_data":false},{"leaderboard":"/sota/action-recognition-on-uav-human","task":"Action Recognition","dataset":"UAV-Human","model":"PMI Sampler","rank_in_archive_order":1,"of":4,"metrics":{"Top 1 Accuracy":"55.0"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}