{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/asynchrony-robust-collaborative-perception-1","title":"Asynchrony-Robust Collaborative Perception via Bird's Eye View Flow","arxiv_id":"2309.16940","date":"2023-09-29","proceeding":"NeurIPS 2023 11","authors":["Sizhe Wei","Yuxi Wei","Yue Hu","Yifan Lu","Yiqi Zhong","Siheng Chen","Ya zhang"],"abstract":"Collaborative perception can substantially boost each agent's perception ability by facilitating communication among multiple agents. However, temporal asynchrony among agents is inevitable in the real world due to communication delays, interruptions, and clock misalignments. This issue causes information mismatch during multi-agent fusion, seriously shaking the foundation of collaboration. To address this issue, we propose CoBEVFlow, an asynchrony-robust collaborative perception system based on bird's eye view (BEV) flow. The key intuition of CoBEVFlow is to compensate motions to align asynchronous collaboration messages sent by multiple agents. To model the motion in a scene, we propose BEV flow, which is a collection of the motion vector corresponding to each spatial location. Based on BEV flow, asynchronous perceptual features can be reassigned to appropriate positions, mitigating the impact of asynchrony. CoBEVFlow has two advantages: (i) CoBEVFlow can handle asynchronous collaboration messages sent at irregular, continuous time stamps without discretization; and (ii) with BEV flow, CoBEVFlow only transports the original perceptual features, instead of generating new perceptual features, avoiding additional noises. To validate CoBEVFlow's efficacy, we create IRregular V2V(IRV2V), the first synthetic collaborative perception dataset with various temporal asynchronies that simulate different real-world scenarios. Extensive experiments conducted on both IRV2V and the real-world dataset DAIR-V2X show that CoBEVFlow consistently outperforms other baselines and is robust in extremely asynchronous settings. The code is available at https://github.com/MediaBrain-SJTU/CoBEVFlow.","url_abs":"https://arxiv.org/abs/2309.16940v2","url_pdf":"https://arxiv.org/pdf/2309.16940v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"asynchrony-robust-collaborative-perception-1","repo_url":"https://github.com/MediaBrain-SJTU/CoBEVFlow","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2309.16940","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.16940"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/MediaBrain-SJTU/CoBEVFlow","reach":null}],"summary":{"ran":9,"ran_fixture":1,"unverified":5},"by_repo_kind":{"official":{"samples":15,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9581f8d7a2626f1e","entry":"Conv3D","repo":"MediaBrain-SJTU/CoBEVFlow","repo_kind":"official","path":"opencood/models/fuse_modules/raindrop_flow.py","file_url":"https://github.com/MediaBrain-SJTU/CoBEVFlow/blob/HEAD/opencood/models/fuse_modules/raindrop_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9581f8d7a2626f1e"}},{"code_sha256_prefix":"977114ba926279fe","entry":"FineTuneFlow","repo":"MediaBrain-SJTU/CoBEVFlow","repo_kind":"official","path":"opencood/models/fuse_modules/raindrop_flow.py","file_url":"https://github.com/MediaBrain-SJTU/CoBEVFlow/blob/HEAD/opencood/models/fuse_modules/raindrop_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"977114ba926279fe"}},{"code_sha256_prefix":"0898a541d6b7a265","entry":"FineTuneFlow_2","repo":"MediaBrain-SJTU/CoBEVFlow","repo_kind":"official","path":"opencood/models/fuse_modules/raindrop_flow.py","file_url":"https://github.com/MediaBrain-SJTU/CoBEVFlow/blob/HEAD/opencood/models/fuse_modules/raindrop_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0898a541d6b7a265"}},{"code_sha256_prefix":"a86dda33c68d31b1","entry":"MaxFusion","repo":"MediaBrain-SJTU/CoBEVFlow","repo_kind":"official","path":"opencood/models/fuse_modules/raindrop_flow.py","file_url":"https://github.com/MediaBrain-SJTU/CoBEVFlow/blob/HEAD/opencood/models/fuse_modules/raindrop_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a86dda33c68d31b1"}},{"code_sha256_prefix":"9b973a68b4839c95","entry":"MotionLSTM","repo":"MediaBrain-SJTU/CoBEVFlow","repo_kind":"official","path":"opencood/models/fuse_modules/raindrop_flow.py","file_url":"https://github.com/MediaBrain-SJTU/CoBEVFlow/blob/HEAD/opencood/models/fuse_modules/raindrop_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9b973a68b4839c95"}},{"code_sha256_prefix":"4a7dceba5ec786fe","entry":"MotionPrediction","repo":"MediaBrain-SJTU/CoBEVFlow","repo_kind":"official","path":"opencood/models/fuse_modules/raindrop_flow.py","file_url":"https://github.com/MediaBrain-SJTU/CoBEVFlow/blob/HEAD/opencood/models/fuse_modules/raindrop_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4a7dceba5ec786fe"}},{"code_sha256_prefix":"bb63e9e21d298214","entry":"STPN_MotionLSTM","repo":"MediaBrain-SJTU/CoBEVFlow","repo_kind":"official","path":"opencood/models/fuse_modules/raindrop_flow.py","file_url":"https://github.com/MediaBrain-SJTU/CoBEVFlow/blob/HEAD/opencood/models/fuse_modules/raindrop_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bb63e9e21d298214"}},{"code_sha256_prefix":"ebff6a1f26d9b016","entry":"StateEstimation","repo":"MediaBrain-SJTU/CoBEVFlow","repo_kind":"official","path":"opencood/models/fuse_modules/raindrop_flow.py","file_url":"https://github.com/MediaBrain-SJTU/CoBEVFlow/blob/HEAD/opencood/models/fuse_modules/raindrop_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ebff6a1f26d9b016"}},{"code_sha256_prefix":"3d695f1959e4d901","entry":"Time_Modulation","repo":"MediaBrain-SJTU/CoBEVFlow","repo_kind":"official","path":"opencood/models/fuse_modules/raindrop_flow.py","file_url":"https://github.com/MediaBrain-SJTU/CoBEVFlow/blob/HEAD/opencood/models/fuse_modules/raindrop_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3d695f1959e4d901"}},{"code_sha256_prefix":"3019b9af698a296f","entry":"warp_affine_simple","repo":"MediaBrain-SJTU/CoBEVFlow","repo_kind":"official","path":"opencood/models/fuse_modules/raindrop_flow.py","file_url":"https://github.com/MediaBrain-SJTU/CoBEVFlow/blob/HEAD/opencood/models/fuse_modules/raindrop_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3019b9af698a296f"}},{"code_sha256_prefix":"0f3512a6d96e8990","entry":"Communication","repo":"MediaBrain-SJTU/CoBEVFlow","repo_kind":"official","path":"opencood/models/fuse_modules/raindrop_flow.py","file_url":"https://github.com/MediaBrain-SJTU/CoBEVFlow/blob/HEAD/opencood/models/fuse_modules/raindrop_flow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f3512a6d96e8990"}},{"code_sha256_prefix":"0c7206596473b49d","entry":"STPN","repo":"MediaBrain-SJTU/CoBEVFlow","repo_kind":"official","path":"opencood/models/fuse_modules/raindrop_flow.py","file_url":"https://github.com/MediaBrain-SJTU/CoBEVFlow/blob/HEAD/opencood/models/fuse_modules/raindrop_flow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0c7206596473b49d"}},{"code_sha256_prefix":"bff03e4bbcbd2b7c","entry":"SyncLSTM","repo":"MediaBrain-SJTU/CoBEVFlow","repo_kind":"official","path":"opencood/models/fuse_modules/raindrop_flow.py","file_url":"https://github.com/MediaBrain-SJTU/CoBEVFlow/blob/HEAD/opencood/models/fuse_modules/raindrop_flow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bff03e4bbcbd2b7c"}},{"code_sha256_prefix":"73363bba425ec840","entry":"get_warped_feature_mask","repo":"MediaBrain-SJTU/CoBEVFlow","repo_kind":"official","path":"opencood/models/fuse_modules/raindrop_flow.py","file_url":"https://github.com/MediaBrain-SJTU/CoBEVFlow/blob/HEAD/opencood/models/fuse_modules/raindrop_flow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"73363bba425ec840"}},{"code_sha256_prefix":"3887b4d00ae7230c","entry":"raindrop_fuse","repo":"MediaBrain-SJTU/CoBEVFlow","repo_kind":"official","path":"opencood/models/fuse_modules/raindrop_flow.py","file_url":"https://github.com/MediaBrain-SJTU/CoBEVFlow/blob/HEAD/opencood/models/fuse_modules/raindrop_flow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3887b4d00ae7230c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}