{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/audiovisual-slowfast-networks-for-video","title":"Audiovisual SlowFast Networks for Video Recognition","arxiv_id":"2001.08740","date":"2020-01-23","proceeding":null,"authors":["Fanyi Xiao","Yong Jae Lee","Kristen Grauman","Jitendra Malik","Christoph Feichtenhofer"],"abstract":"We present Audiovisual SlowFast Networks, an architecture for integrated audiovisual perception. AVSlowFast has Slow and Fast visual pathways that are deeply integrated with a Faster Audio pathway to model vision and sound in a unified representation. We fuse audio and visual features at multiple layers, enabling audio to contribute to the formation of hierarchical audiovisual concepts. To overcome training difficulties that arise from different learning dynamics for audio and visual modalities, we introduce DropPathway, which randomly drops the Audio pathway during training as an effective regularization technique. Inspired by prior studies in neuroscience, we perform hierarchical audiovisual synchronization to learn joint audiovisual features. We report state-of-the-art results on six video action classification and detection datasets, perform detailed ablation studies, and show the generalization of AVSlowFast to learn self-supervised audiovisual features. Code will be made available at: https://github.com/facebookresearch/SlowFast.","url_abs":"https://arxiv.org/abs/2001.08740v2","url_pdf":"https://arxiv.org/pdf/2001.08740v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"audiovisual-slowfast-networks-for-video","repo_url":"https://github.com/facebookresearch/SlowFast","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"audiovisual-slowfast-networks-for-video","repo_url":"https://github.com/2023-MindSpore-4/Code11/tree/main/slowfast","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"audiovisual-slowfast-networks-for-video","repo_url":"https://github.com/code-implementation1/Code8/tree/main/slowfast","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"action-classification","task_name":"Action Classification"},{"task_slug":"video-recognition","task_name":"Video Recognition"}],"methods":[{"method_slug":"avslowfast","method_name":"AVSlowFast"},{"method_slug":"droppathway","method_name":"DropPathway"}],"datasets_introduced":[],"methods_introduced":[{"slug":"avslowfast","name":"AVSlowFast","full_name":"Audiovisual SlowFast Network"},{"slug":"droppathway","name":"DropPathway","full_name":"DropPathway"}],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2001.08740","atlas_url":"https://app.syntology.ai/?focus=2001.08740","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}