{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-sim2real-deep-learning-approach-for-the","title":"A Sim2Real Deep Learning Approach for the Transformation of Images from Multiple Vehicle-Mounted Cameras to a Semantically Segmented Image in Bird's Eye View","arxiv_id":"2005.04078","date":"2020-05-08","proceeding":null,"authors":["Lennart Reiher","Bastian Lampe","Lutz Eckstein"],"abstract":"Accurate environment perception is essential for automated driving. When using monocular cameras, the distance estimation of elements in the environment poses a major challenge. Distances can be more easily estimated when the camera perspective is transformed to a bird's eye view (BEV). For flat surfaces, Inverse Perspective Mapping (IPM) can accurately transform images to a BEV. Three-dimensional objects such as vehicles and vulnerable road users are distorted by this transformation making it difficult to estimate their position relative to the sensor. This paper describes a methodology to obtain a corrected 360{\\deg} BEV image given images from multiple vehicle-mounted cameras. The corrected BEV image is segmented into semantic classes and includes a prediction of occluded areas. The neural network approach does not rely on manually labeled data, but is trained on a synthetic dataset in such a way that it generalizes well to real-world data. By using semantically segmented images as input, we reduce the reality gap between simulated and real-world data and are able to show that our method can be successfully applied in the real world. Extensive experiments conducted on the synthetic data demonstrate the superiority of our approach compared to IPM. Source code and datasets are available at https://github.com/ika-rwth-aachen/Cam2BEV","url_abs":"https://arxiv.org/abs/2005.04078v1","url_pdf":"https://arxiv.org/pdf/2005.04078v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-sim2real-deep-learning-approach-for-the","repo_url":"https://github.com/ika-rwth-aachen/Cam2BEV","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"a-sim2real-deep-learning-approach-for-the","repo_url":"https://github.com/ika-rwth-aachen/etsi_its_messages","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"bird-view-synthesis","task_name":"Bird View Synthesis"},{"task_slug":"cross-view-image-to-image-translation","task_name":"Cross-View Image-to-Image Translation"},{"task_slug":"image-stitching","task_name":"Image Stitching"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"aspp","method_name":"ASPP"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"concatenated-skip-connection","method_name":"Concatenated Skip Connection"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"deeplabv3","method_name":"DeepLabv3"},{"method_slug":"depthwise-convolution","method_name":"Depthwise Convolution"},{"method_slug":"depthwise-separable-convolution","method_name":"Depthwise Separable Convolution"},{"method_slug":"dilated-convolution","method_name":"Dilated Convolution"},{"method_slug":"inverted-residual-block","method_name":"Inverted Residual Block"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"pointwise-convolution","method_name":"Pointwise Convolution"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"spatial-pyramid-pooling","method_name":"Spatial Pyramid Pooling"},{"method_slug":"spatial-transformer","method_name":"Spatial Transformer"},{"method_slug":"unetxst","method_name":"uNetXST"}],"datasets_introduced":[{"slug":"cam2bev","name":"Cam2BEV","full_name":""}],"methods_introduced":[{"slug":"unetxst","name":"uNetXST","full_name":"uNetXST"}],"results":[{"leaderboard":"/sota/cross-view-image-to-image-translation-on-6","task":"Cross-View Image-to-Image Translation","dataset":"Cam2BEV","model":"uNetXST","rank_in_archive_order":1,"of":1,"metrics":{"Mean IoU":"71.92"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-cam2bev","task":"Semantic Segmentation","dataset":"Cam2BEV","model":"uNetXST","rank_in_archive_order":1,"of":1,"metrics":{"Mean IoU":"71.92"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2005.04078","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2005.04078"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ika-rwth-aachen/Cam2BEV","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ika-rwth-aachen/etsi_its_messages","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":6},"by_repo_kind":{"official":{"samples":5,"ran":0,"repositories":1},"listed":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2395b4c02267027a","entry":"abspath","repo":"ika-rwth-aachen/Cam2BEV","repo_kind":"official","path":"model/utils.py","file_url":"https://github.com/ika-rwth-aachen/Cam2BEV/blob/HEAD/model/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2395b4c02267027a"}},{"code_sha256_prefix":"ddfa1f70e251fee1","entry":"decoder","repo":"ika-rwth-aachen/Cam2BEV","repo_kind":"official","path":"model/architecture/uNetXST.py","file_url":"https://github.com/ika-rwth-aachen/Cam2BEV/blob/HEAD/model/architecture/uNetXST.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ddfa1f70e251fee1"}},{"code_sha256_prefix":"c0039d5cb7a69174","entry":"encoder","repo":"ika-rwth-aachen/Cam2BEV","repo_kind":"official","path":"model/architecture/uNetXST.py","file_url":"https://github.com/ika-rwth-aachen/Cam2BEV/blob/HEAD/model/architecture/uNetXST.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c0039d5cb7a69174"}},{"code_sha256_prefix":"32c0a561973913e5","entry":"get_files_in_folder","repo":"ika-rwth-aachen/Cam2BEV","repo_kind":"official","path":"model/utils.py","file_url":"https://github.com/ika-rwth-aachen/Cam2BEV/blob/HEAD/model/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"32c0a561973913e5"}},{"code_sha256_prefix":"0a11a74a64c6dd13","entry":"get_t_its","repo":"ika-rwth-aachen/etsi_its_messages","repo_kind":"listed","path":"etsi_its_msgs_utils/samples/utils.py","file_url":"https://github.com/ika-rwth-aachen/etsi_its_messages/blob/HEAD/etsi_its_msgs_utils/samples/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0a11a74a64c6dd13"}},{"code_sha256_prefix":"6965945ec615f691","entry":"load_module","repo":"ika-rwth-aachen/Cam2BEV","repo_kind":"official","path":"model/utils.py","file_url":"https://github.com/ika-rwth-aachen/Cam2BEV/blob/HEAD/model/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6965945ec615f691"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}