{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/3d-dynamic-scene-graphs-actionable-spatial","title":"3D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans","arxiv_id":"2002.06289","date":"2020-02-15","proceeding":null,"authors":["Antoni Rosinol","Arjun Gupta","Marcus Abate","Jingnan Shi","Luca Carlone"],"abstract":"We present a unified representation for actionable spatial perception: 3D Dynamic Scene Graphs. Scene graphs are directed graphs where nodes represent entities in the scene (e.g. objects, walls, rooms), and edges represent relations (e.g. inclusion, adjacency) among nodes. Dynamic scene graphs (DSGs) extend this notion to represent dynamic scenes with moving agents (e.g. humans, robots), and to include actionable information that supports planning and decision-making (e.g. spatio-temporal relations, topology at different levels of abstraction). Our second contribution is to provide the first fully automatic Spatial PerceptIon eNgine(SPIN) to build a DSG from visual-inertial data. We integrate state-of-the-art techniques for object and human detection and pose estimation, and we describe how to robustly infer object, robot, and human nodes in crowded scenes. To the best of our knowledge, this is the first paper that reconciles visual-inertial SLAM and dense human mesh tracking. Moreover, we provide algorithms to obtain hierarchical representations of indoor environments (e.g. places, structures, rooms) and their relations. Our third contribution is to demonstrate the proposed spatial perception engine in a photo-realistic Unity-based simulator, where we assess its robustness and expressiveness. Finally, we discuss the implications of our proposal on modern robotics applications. 3D Dynamic Scene Graphs can have a profound impact on planning and decision-making, human-robot interaction, long-term autonomy, and scene prediction. A video abstract is available at https://youtu.be/SWbofjhyPzI","url_abs":"https://arxiv.org/abs/2002.06289v2","url_pdf":"https://arxiv.org/pdf/2002.06289v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"3d-dynamic-scene-graphs-actionable-spatial","repo_url":"https://github.com/MIT-SPARK/Kimera","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"3d-dynamic-scene-graphs-actionable-spatial","repo_url":"https://github.com/MIT-SPARK/Kimera-VIO-Evaluation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"3d-dynamic-scene-graphs-actionable-spatial","repo_url":"https://github.com/ToniRV/Kimera-VIO-Evaluation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"3d-reconstruction","task_name":"3D Reconstruction"},{"task_slug":"human-detection","task_name":"Human Detection"},{"task_slug":"robot-task-planning","task_name":"Robot Task Planning"},{"task_slug":"simultaneous-localization-and-mapping","task_name":"Simultaneous Localization and Mapping"},{"task_slug":"unity","task_name":"Unity"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2002.06289","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2002.06289"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MIT-SPARK/Kimera","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MIT-SPARK/Kimera-VIO-Evaluation","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ToniRV/Kimera-VIO-Evaluation","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":4},"by_repo_kind":{"listed":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7877d692dc622ad3","entry":"find_submissions","repo":"MIT-SPARK/Kimera-VIO-Evaluation","repo_kind":"listed","path":"evaluation/go_seek_evaluation.py","file_url":"https://github.com/MIT-SPARK/Kimera-VIO-Evaluation/blob/HEAD/evaluation/go_seek_evaluation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7877d692dc622ad3"}},{"code_sha256_prefix":"3d95f1c3d29f54c2","entry":"find_submissions","repo":"MIT-SPARK/Kimera-VIO-Evaluation","repo_kind":"listed","path":"evaluation/uHumans_evaluation.py","file_url":"https://github.com/MIT-SPARK/Kimera-VIO-Evaluation/blob/HEAD/evaluation/uHumans_evaluation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3d95f1c3d29f54c2"}},{"code_sha256_prefix":"b34d6ebc2acb186d","entry":"get_pipeline_times","repo":"MIT-SPARK/Kimera-VIO-Evaluation","repo_kind":"listed","path":"plotting_tools/display_timing.py","file_url":"https://github.com/MIT-SPARK/Kimera-VIO-Evaluation/blob/HEAD/plotting_tools/display_timing.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b34d6ebc2acb186d"}},{"code_sha256_prefix":"092fb93f1a94ce62","entry":"write_yaml_params","repo":"MIT-SPARK/Kimera-VIO-Evaluation","repo_kind":"listed","path":"evaluation/regression_tests.py","file_url":"https://github.com/MIT-SPARK/Kimera-VIO-Evaluation/blob/HEAD/evaluation/regression_tests.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"092fb93f1a94ce62"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}