{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multi-attention-networks-for-temporal","title":"Multi-attention Networks for Temporal Localization of Video-level Labels","arxiv_id":"1911.06866","date":"2019-11-15","proceeding":null,"authors":["Lijun Zhang","Srinath Nizampatnam","Ahana Gangopadhyay","Marcos V. Conde"],"abstract":"Temporal localization remains an important challenge in video understanding. In this work, we present our solution to the 3rd YouTube-8M Video Understanding Challenge organized by Google Research. Participants were required to build a segment-level classifier using a large-scale training data set with noisy video-level labels and a relatively small-scale validation data set with accurate segment-level labels. We formulated the problem as a multiple instance multi-label learning and developed an attention-based mechanism to selectively emphasize the important frames by attention weights. The model performance is further improved by constructing multiple sets of attention networks. We further fine-tuned the model using the segment-level data set. Our final model consists of an ensemble of attention/multi-attention networks, deep bag of frames models, recurrent neural networks and convolutional neural networks. It ranked 13th on the private leader board and stands out for its efficient usage of resources.","url_abs":"https://arxiv.org/abs/1911.06866v1","url_pdf":"https://arxiv.org/pdf/1911.06866v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multi-attention-networks-for-temporal","repo_url":"https://github.com/mv-lab/youtube8m-19","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok"}}],"tasks":[{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"action-recognition","task_name":"Temporal Action Localization"},{"task_slug":"temporal-localization","task_name":"Temporal Localization"},{"task_slug":"video-captioning","task_name":"Video Captioning"},{"task_slug":"video-understanding","task_name":"Video Understanding"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"gru","method_name":"GRU"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-attention-network","method_name":"Multi-Attention Network"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}