{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/camliflow-bidirectional-camera-lidar-fusion","title":"CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimation","arxiv_id":"2111.10502","date":"2021-11-20","proceeding":"CVPR 2022 1","authors":["Haisong Liu","Tao Lu","Yihui Xu","Jia Liu","Wenjie Li","Lijun Chen"],"abstract":"In this paper, we study the problem of jointly estimating the optical flow and scene flow from synchronized 2D and 3D data. Previous methods either employ a complex pipeline that splits the joint task into independent stages, or fuse 2D and 3D information in an \"early-fusion\" or \"late-fusion\" manner. Such one-size-fits-all approaches suffer from a dilemma of failing to fully utilize the characteristic of each modality or to maximize the inter-modality complementarity. To address the problem, we propose a novel end-to-end framework, called CamLiFlow. It consists of 2D and 3D branches with multiple bidirectional connections between them in specific layers. Different from previous work, we apply a point-based 3D branch to better extract the geometric features and design a symmetric learnable operator to fuse dense image features and sparse point features. Experiments show that CamLiFlow achieves better performance with fewer parameters. Our method ranks 1st on the KITTI Scene Flow benchmark, outperforming the previous art with 1/7 parameters. Code is available at https://github.com/MCG-NJU/CamLiFlow.","url_abs":"https://arxiv.org/abs/2111.10502v4","url_pdf":"https://arxiv.org/pdf/2111.10502v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"camliflow-bidirectional-camera-lidar-fusion","repo_url":"https://github.com/mcg-nju/camliflow","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"optical-flow-estimation","task_name":"Optical Flow Estimation"},{"task_slug":"scene-flow-estimation","task_name":"Scene Flow Estimation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/scene-flow-estimation-on-spring","task":"Scene Flow Estimation","dataset":"Spring","model":"CamLiFlow (F)","rank_in_archive_order":3,"of":6,"metrics":{"1px total":"50.083"},"uses_additional_data":false},{"leaderboard":"/sota/scene-flow-estimation-on-spring","task":"Scene Flow Estimation","dataset":"Spring","model":"CamLiFlow (K)","rank_in_archive_order":6,"of":6,"metrics":{"1px total":"85.310"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2111.10502","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}