{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ma2cl-masked-attentive-contrastive-learning","title":"MA2CL:Masked Attentive Contrastive Learning for Multi-Agent Reinforcement Learning","arxiv_id":"2306.02006","date":"2023-06-03","proceeding":null,"authors":["Haolin Song","Mingxiao Feng","Wengang Zhou","Houqiang Li"],"abstract":"Recent approaches have utilized self-supervised auxiliary tasks as representation learning to improve the performance and sample efficiency of vision-based reinforcement learning algorithms in single-agent settings. However, in multi-agent reinforcement learning (MARL), these techniques face challenges because each agent only receives partial observation from an environment influenced by others, resulting in correlated observations in the agent dimension. So it is necessary to consider agent-level information in representation learning for MARL. In this paper, we propose an effective framework called \\textbf{M}ulti-\\textbf{A}gent \\textbf{M}asked \\textbf{A}ttentive \\textbf{C}ontrastive \\textbf{L}earning (MA2CL), which encourages learning representation to be both temporal and agent-level predictive by reconstructing the masked agent observation in latent space. Specifically, we use an attention reconstruction model for recovering and the model is trained via contrastive learning. MA2CL allows better utilization of contextual information at the agent level, facilitating the training of MARL agents for cooperation tasks. Extensive experiments demonstrate that our method significantly improves the performance and sample efficiency of different MARL algorithms and outperforms other methods in various vision-based and state-based scenarios. Our code can be found in \\url{https://github.com/ustchlsong/MA2CL}","url_abs":"https://arxiv.org/abs/2306.02006v1","url_pdf":"https://arxiv.org/pdf/2306.02006v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ma2cl-masked-attentive-contrastive-learning","repo_url":"https://github.com/ustchlsong/ma2cl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"multi-agent-reinforcement-learning","task_name":"Multi-agent Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.02006","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.02006"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ustchlsong/ma2cl","reach":null}],"summary":{"ran":5,"ran_draft_wrong":1,"ran_fixture":1,"unverified":3},"by_repo_kind":{"official":{"samples":10,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"858291b5d986d082","entry":"CNNLayer","repo":"ustchlsong/ma2cl","repo_kind":"official","path":"algorithms/ma_transformer.py","file_url":"https://github.com/ustchlsong/ma2cl/blob/HEAD/algorithms/ma_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"858291b5d986d082"}},{"code_sha256_prefix":"2e6742e3206a6c46","entry":"DecodeBlock","repo":"ustchlsong/ma2cl","repo_kind":"official","path":"algorithms/ma_transformer.py","file_url":"https://github.com/ustchlsong/ma2cl/blob/HEAD/algorithms/ma_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2e6742e3206a6c46"}},{"code_sha256_prefix":"ca5abd37196a0511","entry":"EncodeBlock","repo":"ustchlsong/ma2cl","repo_kind":"official","path":"algorithms/ma_transformer.py","file_url":"https://github.com/ustchlsong/ma2cl/blob/HEAD/algorithms/ma_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ca5abd37196a0511"}},{"code_sha256_prefix":"f9358cb500fd1a12","entry":"Encoder","repo":"ustchlsong/ma2cl","repo_kind":"official","path":"algorithms/ma_transformer.py","file_url":"https://github.com/ustchlsong/ma2cl/blob/HEAD/algorithms/ma_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f9358cb500fd1a12"}},{"code_sha256_prefix":"4a5fcc8f9eab1543","entry":"SelfAttention","repo":"ustchlsong/ma2cl","repo_kind":"official","path":"algorithms/ma_transformer.py","file_url":"https://github.com/ustchlsong/ma2cl/blob/HEAD/algorithms/ma_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4a5fcc8f9eab1543"}},{"code_sha256_prefix":"8746e100c8735d36","entry":"discrete_autoregreesive_act","repo":"ustchlsong/ma2cl","repo_kind":"official","path":"algorithms/ma_transformer.py","file_url":"https://github.com/ustchlsong/ma2cl/blob/HEAD/algorithms/ma_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8746e100c8735d36"}},{"code_sha256_prefix":"12aa1ad4303a9073","entry":"discrete_parallel_act","repo":"ustchlsong/ma2cl","repo_kind":"official","path":"algorithms/ma_transformer.py","file_url":"https://github.com/ustchlsong/ma2cl/blob/HEAD/algorithms/ma_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"12aa1ad4303a9073"}},{"code_sha256_prefix":"14d0620195ae45fe","entry":"Decoder","repo":"ustchlsong/ma2cl","repo_kind":"official","path":"algorithms/ma_transformer.py","file_url":"https://github.com/ustchlsong/ma2cl/blob/HEAD/algorithms/ma_transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"14d0620195ae45fe"}},{"code_sha256_prefix":"33ea553ff4f41e56","entry":"MultiAgentTransformer","repo":"ustchlsong/ma2cl","repo_kind":"official","path":"algorithms/ma_transformer.py","file_url":"https://github.com/ustchlsong/ma2cl/blob/HEAD/algorithms/ma_transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"33ea553ff4f41e56"}},{"code_sha256_prefix":"90c40764114ee949","entry":"continuous_autoregreesive_act","repo":"ustchlsong/ma2cl","repo_kind":"official","path":"algorithms/ma_transformer.py","file_url":"https://github.com/ustchlsong/ma2cl/blob/HEAD/algorithms/ma_transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"90c40764114ee949"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}