{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/190511377","title":"FlightGoggles: A Modular Framework for Photorealistic Camera, Exteroceptive Sensor, and Dynamics Simulation","arxiv_id":"1905.11377","date":"2019-05-27","proceeding":null,"authors":["Winter Guerra","Ezra Tal","Varun Murali","Gilhyun Ryou","Sertac Karaman"],"abstract":"FlightGoggles is a photorealistic sensor simulator for perception-driven robotic vehicles. The key contributions of FlightGoggles are twofold. First, FlightGoggles provides photorealistic exteroceptive sensor simulation using graphics assets generated with photogrammetry. Second, it provides the ability to combine (i) synthetic exteroceptive measurements generated in silico in real time and (ii) vehicle dynamics and proprioceptive measurements generated in motio by vehicle(s) in a motion-capture facility. FlightGoggles is capable of simulating a virtual-reality environment around autonomous vehicle(s). While a vehicle is in flight in the FlightGoggles virtual reality environment, exteroceptive sensors are rendered synthetically in real time while all complex extrinsic dynamics are generated organically through the natural interactions of the vehicle. The FlightGoggles framework allows for researchers to accelerate development by circumventing the need to estimate complex and hard-to-model interactions such as aerodynamics, motor mechanics, battery electrochemistry, and behavior of other agents. The ability to perform vehicle-in-the-loop experiments with photorealistic exteroceptive sensor simulation facilitates novel research directions involving, e.g., fast and agile autonomous flight in obstacle-rich environments, safe human interaction, and flexible sensor selection. FlightGoggles has been utilized as the main test for selecting nine teams that will advance in the AlphaPilot autonomous drone racing challenge. We survey approaches and results from the top AlphaPilot teams, which may be of independent interest.","url_abs":"https://arxiv.org/abs/1905.11377v2","url_pdf":"https://arxiv.org/pdf/1905.11377v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"190511377","repo_url":"https://github.com/AgileDrones/FlightGoggles","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"190511377","repo_url":"https://github.com/mit-aera/FlightGoggles","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"190511377","repo_url":"https://github.com/mit-aera/multicopterDynamicsSim","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"190511377","repo_url":"https://github.com/mit-aera/pyFlightGoggles","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"190511377","repo_url":"https://github.com/mit-fast/FlightGoggles","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"190511377","repo_url":"https://github.com/mit-fast/multicopterDynamicsSim","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1905.11377","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1905.11377"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mit-aera/multicopterDynamicsSim","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/AgileDrones/FlightGoggles","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mit-aera/FlightGoggles","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mit-fast/FlightGoggles","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mit-aera/pyFlightGoggles","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mit-fast/multicopterDynamicsSim","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":6},"by_repo_kind":{"listed":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e146c08faaf39eb4","entry":"catch_exception","repo":"mit-aera/pyFlightGoggles","repo_kind":"listed","path":"flightgoggles/env.py","file_url":"https://github.com/mit-aera/pyFlightGoggles/blob/HEAD/flightgoggles/env.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e146c08faaf39eb4"}},{"code_sha256_prefix":"34afea7d0fc19cb0","entry":"for_all_methods","repo":"mit-aera/pyFlightGoggles","repo_kind":"listed","path":"flightgoggles/env.py","file_url":"https://github.com/mit-aera/pyFlightGoggles/blob/HEAD/flightgoggles/env.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"34afea7d0fc19cb0"}},{"code_sha256_prefix":"02b7fa755c76e212","entry":"quat_wx2xw","repo":"mit-aera/pyFlightGoggles","repo_kind":"listed","path":"flightgoggles/utils.py","file_url":"https://github.com/mit-aera/pyFlightGoggles/blob/HEAD/flightgoggles/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"02b7fa755c76e212"}},{"code_sha256_prefix":"9d7ea851468b86c4","entry":"quat_xw2wx","repo":"mit-aera/pyFlightGoggles","repo_kind":"listed","path":"flightgoggles/utils.py","file_url":"https://github.com/mit-aera/pyFlightGoggles/blob/HEAD/flightgoggles/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9d7ea851468b86c4"}},{"code_sha256_prefix":"fb846a9ef0f12b55","entry":"unwrap","repo":"mit-aera/pyFlightGoggles","repo_kind":"listed","path":"flightgoggles/utils.py","file_url":"https://github.com/mit-aera/pyFlightGoggles/blob/HEAD/flightgoggles/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fb846a9ef0f12b55"}},{"code_sha256_prefix":"c00800722c7d1043","entry":"update_pose_with_bias","repo":"mit-aera/pyFlightGoggles","repo_kind":"listed","path":"flightgoggles/model.py","file_url":"https://github.com/mit-aera/pyFlightGoggles/blob/HEAD/flightgoggles/model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c00800722c7d1043"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}