{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mm-pyramid-multimodal-pyramid-attentional","title":"MM-Pyramid: Multimodal Pyramid Attentional Network for Audio-Visual Event Localization and Video Parsing","arxiv_id":"2111.12374","date":"2021-11-24","proceeding":null,"authors":["Jiashuo Yu","Ying Cheng","Rui-Wei Zhao","Rui Feng","Yuejie Zhang"],"abstract":"Recognizing and localizing events in videos is a fundamental task for video understanding. Since events may occur in auditory and visual modalities, multimodal detailed perception is essential for complete scene comprehension. Most previous works attempted to analyze videos from a holistic perspective. However, they do not consider semantic information at multiple scales, which makes the model difficult to localize events in different lengths. In this paper, we present a Multimodal Pyramid Attentional Network (\\textbf{MM-Pyramid}) for event localization. Specifically, we first propose the attentive feature pyramid module. This module captures temporal pyramid features via several stacking pyramid units, each of them is composed of a fixed-size attention block and dilated convolution block. We also design an adaptive semantic fusion module, which leverages a unit-level attention block and a selective fusion block to integrate pyramid features interactively. Extensive experiments on audio-visual event localization and weakly-supervised audio-visual video parsing tasks verify the effectiveness of our approach.","url_abs":"https://arxiv.org/abs/2111.12374v2","url_pdf":"https://arxiv.org/pdf/2111.12374v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mm-pyramid-multimodal-pyramid-attentional","repo_url":"https://github.com/JustinYuu/MM_Pyramid","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"video-understanding","task_name":"Video Understanding"},{"task_slug":"audio-visual-event-localization","task_name":"audio-visual event localization"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dilated-convolution","method_name":"Dilated Convolution"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2111.12374","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2111.12374"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/JustinYuu/MM_Pyramid","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"unverified":5},"by_repo_kind":{"official":{"samples":6,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d2d810cddae9f875","entry":"clones","repo":"JustinYuu/MM_Pyramid","repo_kind":"official","path":"nets/Transformer.py","file_url":"https://github.com/JustinYuu/MM_Pyramid/blob/HEAD/nets/Transformer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d2d810cddae9f875"}},{"code_sha256_prefix":"91bad2947750b184","entry":"F1","repo":"JustinYuu/MM_Pyramid","repo_kind":"official","path":"utils/eval_metrics.py","file_url":"https://github.com/JustinYuu/MM_Pyramid/blob/HEAD/utils/eval_metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"91bad2947750b184"}},{"code_sha256_prefix":"1531200fadc3e3dd","entry":"Precision","repo":"JustinYuu/MM_Pyramid","repo_kind":"official","path":"utils/eval_metrics.py","file_url":"https://github.com/JustinYuu/MM_Pyramid/blob/HEAD/utils/eval_metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1531200fadc3e3dd"}},{"code_sha256_prefix":"c30d80d42efda95a","entry":"Recall","repo":"JustinYuu/MM_Pyramid","repo_kind":"official","path":"utils/eval_metrics.py","file_url":"https://github.com/JustinYuu/MM_Pyramid/blob/HEAD/utils/eval_metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c30d80d42efda95a"}},{"code_sha256_prefix":"d549060ffa0a0179","entry":"attention","repo":"JustinYuu/MM_Pyramid","repo_kind":"official","path":"nets/Transformer.py","file_url":"https://github.com/JustinYuu/MM_Pyramid/blob/HEAD/nets/Transformer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d549060ffa0a0179"}},{"code_sha256_prefix":"72ea325e53e5b472","entry":"ids_to_multinomial","repo":"JustinYuu/MM_Pyramid","repo_kind":"official","path":"dataloader_avvp.py","file_url":"https://github.com/JustinYuu/MM_Pyramid/blob/HEAD/dataloader_avvp.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"72ea325e53e5b472"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}