{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/2408-01952","title":"CACE-Net: Co-guidance Attention and Contrastive Enhancement for Effective Audio-Visual Event Localization","arxiv_id":"2408.01952","date":"2024-08-04","proceeding":null,"authors":["Xiang He","Xiangxi Liu","Yang Li","Dongcheng Zhao","Guobin Shen","Qingqun Kong","Xin Yang","Yi Zeng"],"abstract":"The audio-visual event localization task requires identifying concurrent visual and auditory events from unconstrained videos within a network model, locating them, and classifying their category. The efficient extraction and integration of audio and visual modal information have always been challenging in this field. In this paper, we introduce CACE-Net, which differs from most existing methods that solely use audio signals to guide visual information. We propose an audio-visual co-guidance attention mechanism that allows for adaptive bi-directional cross-modal attentional guidance between audio and visual information, thus reducing inconsistencies between modalities. Moreover, we have observed that existing methods have difficulty distinguishing between similar background and event and lack the fine-grained features for event classification. Consequently, we employ background-event contrast enhancement to increase the discrimination of fused feature and fine-tuned pre-trained model to extract more refined and discernible features from complex multimodal inputs. Specifically, we have enhanced the model's ability to discern subtle differences between event and background and improved the accuracy of event classification in our model. Experiments on the AVE dataset demonstrate that CACE-Net sets a new benchmark in the audio-visual event localization task, proving the effectiveness of our proposed methods in handling complex multimodal learning and event localization in unconstrained videos. Code is available at https://github.com/Brain-Cog-Lab/CACE-Net.","url_abs":"https://arxiv.org/abs/2408.01952v1","url_pdf":"https://arxiv.org/pdf/2408.01952v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"2408-01952","repo_url":"https://github.com/brain-cog-lab/cace-net","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"audio-visual-event-localization","task_name":"audio-visual event localization"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2408.01952","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2408.01952"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/brain-cog-lab/cace-net","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":7,"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":9,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c34be962e1cee0c4","entry":"aggregate_data","repo":"brain-cog-lab/cace-net","repo_kind":"official","path":"CACE/plt_fig.py","file_url":"https://github.com/brain-cog-lab/cace-net/blob/HEAD/CACE/plt_fig.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c34be962e1cee0c4"}},{"code_sha256_prefix":"7f8930ec08c074cc","entry":"compute_accuracy_supervised","repo":"brain-cog-lab/cace-net","repo_kind":"official","path":"CACE/supv_main.py","file_url":"https://github.com/brain-cog-lab/cace-net/blob/HEAD/CACE/supv_main.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7f8930ec08c074cc"}},{"code_sha256_prefix":"403277d0bfbb477b","entry":"find_classes","repo":"brain-cog-lab/cace-net","repo_kind":"official","path":"CACE/braincog/datasets/TinyImageNet.py","file_url":"https://github.com/brain-cog-lab/cace-net/blob/HEAD/CACE/braincog/datasets/TinyImageNet.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"403277d0bfbb477b"}},{"code_sha256_prefix":"5ded0879a886f3c4","entry":"make_dataset","repo":"brain-cog-lab/cace-net","repo_kind":"official","path":"CACE/braincog/datasets/TinyImageNet.py","file_url":"https://github.com/brain-cog-lab/cace-net/blob/HEAD/CACE/braincog/datasets/TinyImageNet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5ded0879a886f3c4"}},{"code_sha256_prefix":"e94bfa0774663f4e","entry":"normlize","repo":"brain-cog-lab/cace-net","repo_kind":"official","path":"CACE/att_vis.py","file_url":"https://github.com/brain-cog-lab/cace-net/blob/HEAD/CACE/att_vis.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e94bfa0774663f4e"}},{"code_sha256_prefix":"8ccf5c65f536b8a8","entry":"parse_log","repo":"brain-cog-lab/cace-net","repo_kind":"official","path":"CACE/plt_fig.py","file_url":"https://github.com/brain-cog-lab/cace-net/blob/HEAD/CACE/plt_fig.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8ccf5c65f536b8a8"}},{"code_sha256_prefix":"8788abf86694000a","entry":"to_percent","repo":"brain-cog-lab/cace-net","repo_kind":"official","path":"CACE/plt_fig.py","file_url":"https://github.com/brain-cog-lab/cace-net/blob/HEAD/CACE/plt_fig.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8788abf86694000a"}},{"code_sha256_prefix":"a16fbfc15b6fdb82","entry":"video_frame_sample","repo":"brain-cog-lab/cace-net","repo_kind":"official","path":"CACE/att_vis.py","file_url":"https://github.com/brain-cog-lab/cace-net/blob/HEAD/CACE/att_vis.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a16fbfc15b6fdb82"}},{"code_sha256_prefix":"f0c6432c830d10d7","entry":"create_heatmap","repo":"brain-cog-lab/cace-net","repo_kind":"official","path":"CACE/att_vis.py","file_url":"https://github.com/brain-cog-lab/cace-net/blob/HEAD/CACE/att_vis.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f0c6432c830d10d7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}