{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/playing-for-data-ground-truth-from-computer","title":"Playing for Data: Ground Truth from Computer Games","arxiv_id":"1608.02192","date":"2016-08-07","proceeding":null,"authors":["Stephan R. Richter","Vibhav Vineet","Stefan Roth","Vladlen Koltun"],"abstract":"Recent progress in computer vision has been driven by high-capacity models\ntrained on large datasets. Unfortunately, creating large datasets with\npixel-level labels has been extremely costly due to the amount of human effort\nrequired. In this paper, we present an approach to rapidly creating\npixel-accurate semantic label maps for images extracted from modern computer\ngames. Although the source code and the internal operation of commercial games\nare inaccessible, we show that associations between image patches can be\nreconstructed from the communication between the game and the graphics\nhardware. This enables rapid propagation of semantic labels within and across\nimages synthesized by the game, with no access to the source code or the\ncontent. We validate the presented approach by producing dense pixel-level\nsemantic annotations for 25 thousand images synthesized by a photorealistic\nopen-world computer game. Experiments on semantic segmentation datasets show\nthat using the acquired data to supplement real-world images significantly\nincreases accuracy and that the acquired data enables reducing the amount of\nhand-labeled real-world data: models trained with game data and just 1/3 of the\nCamVid training set outperform models trained on the complete CamVid training\nset.","url_abs":"http://arxiv.org/abs/1608.02192v1","url_pdf":"http://arxiv.org/pdf/1608.02192v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"playing-for-data-ground-truth-from-computer","repo_url":"https://bitbucket.org/visinf/projects-2016-playing-for-data","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null},{"paper_slug":"playing-for-data-ground-truth-from-computer","repo_url":"https://github.com/slevin48/gta","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"GPL-3.0"}}],"tasks":[{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[],"datasets_introduced":[{"slug":"gta5","name":"GTA5","full_name":"Grand Theft Auto 5"}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1608.02192","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}