{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/building-generalizable-agents-with-a","title":"Building Generalizable Agents with a Realistic and Rich 3D Environment","arxiv_id":"1801.02209","date":"2018-01-07","proceeding":"ICLR 2018 1","authors":["Yi Wu","Yuxin Wu","Georgia Gkioxari","Yuandong Tian"],"abstract":"Teaching an agent to navigate in an unseen 3D environment is a challenging\ntask, even in the event of simulated environments. To generalize to unseen\nenvironments, an agent needs to be robust to low-level variations (e.g. color,\ntexture, object changes), and also high-level variations (e.g. layout changes\nof the environment). To improve overall generalization, all types of variations\nin the environment have to be taken under consideration via different level of\ndata augmentation steps. To this end, we propose House3D, a rich, extensible\nand efficient environment that contains 45,622 human-designed 3D scenes of\nvisually realistic houses, ranging from single-room studios to multi-storied\nhouses, equipped with a diverse set of fully labeled 3D objects, textures and\nscene layouts, based on the SUNCG dataset (Song et.al.). The diversity in\nHouse3D opens the door towards scene-level augmentation, while the label-rich\nnature of House3D enables us to inject pixel- & task-level augmentations such\nas domain randomization (Toubin et. al.) and multi-task training. Using a\nsubset of houses in House3D, we show that reinforcement learning agents trained\nwith an enhancement of different levels of augmentations perform much better in\nunseen environments than our baselines with raw RGB input by over 8% in terms\nof navigation success rate. House3D is publicly available at\nhttp://github.com/facebookresearch/House3D.","url_abs":"http://arxiv.org/abs/1801.02209v2","url_pdf":"http://arxiv.org/pdf/1801.02209v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"building-generalizable-agents-with-a","repo_url":"https://github.com/facebookresearch/House3D","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"building-generalizable-agents-with-a","repo_url":"https://github.com/abhshkdz/House3D","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"building-generalizable-agents-with-a","repo_url":"https://github.com/jxwuyi/House3D","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"building-generalizable-agents-with-a","repo_url":"https://github.com/jxwuyi/HouseNavAgent","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"building-generalizable-agents-with-a","repo_url":"https://github.com/kibeomKim/House3D_baseline","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"}],"methods":[],"datasets_introduced":[{"slug":"house3d-environment","name":"House3D Environment","full_name":""}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1801.02209","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}