{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/geometry-free-view-synthesis-transformers-and","title":"Geometry-Free View Synthesis: Transformers and no 3D Priors","arxiv_id":"2104.07652","date":"2021-04-15","proceeding":"ICCV 2021 10","authors":["Robin Rombach","Patrick Esser","Björn Ommer"],"abstract":"Is a geometric model required to synthesize novel views from a single image? Being bound to local convolutions, CNNs need explicit 3D biases to model geometric transformations. In contrast, we demonstrate that a transformer-based model can synthesize entirely novel views without any hand-engineered 3D biases. This is achieved by (i) a global attention mechanism for implicitly learning long-range 3D correspondences between source and target views, and (ii) a probabilistic formulation necessary to capture the ambiguity inherent in predicting novel views from a single image, thereby overcoming the limitations of previous approaches that are restricted to relatively small viewpoint changes. We evaluate various ways to integrate 3D priors into a transformer architecture. However, our experiments show that no such geometric priors are required and that the transformer is capable of implicitly learning 3D relationships between images. Furthermore, this approach outperforms the state of the art in terms of visual quality while covering the full distribution of possible realizations. Code is available at https://git.io/JOnwn","url_abs":"https://arxiv.org/abs/2104.07652v2","url_pdf":"https://arxiv.org/pdf/2104.07652v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"geometry-free-view-synthesis-transformers-and","repo_url":"https://github.com/CompVis/geometry-free-view-synthesis","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"novel-view-synthesis","task_name":"Novel View Synthesis"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/novel-view-synthesis-on-acid","task":"Novel View Synthesis","dataset":"ACID","model":"impl.-nodepth","rank_in_archive_order":1,"of":3,"metrics":{"FID":"42.88"},"uses_additional_data":false},{"leaderboard":"/sota/novel-view-synthesis-on-acid","task":"Novel View Synthesis","dataset":"ACID","model":"hybrid","rank_in_archive_order":2,"of":3,"metrics":{"NLL":"5.341","PSIM":"2.83","PSNR":"15.54"},"uses_additional_data":false},{"leaderboard":"/sota/novel-view-synthesis-on-acid","task":"Novel View Synthesis","dataset":"ACID","model":"impl.-catdepth","rank_in_archive_order":3,"of":3,"metrics":{"SSIM":"0.42"},"uses_additional_data":false},{"leaderboard":"/sota/novel-view-synthesis-on-realestate10k","task":"Novel View Synthesis","dataset":"RealEstate10K","model":"Hybrid","rank_in_archive_order":1,"of":2,"metrics":{"FID":"48.84","PSNR":"12.51"},"uses_additional_data":false},{"leaderboard":"/sota/novel-view-synthesis-on-realestate10k","task":"Novel View Synthesis","dataset":"RealEstate10K","model":"Impl.-depth","rank_in_archive_order":2,"of":2,"metrics":{"NLL":"4.836","PSIM":"3.05","SSIM":"0.44"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2104.07652","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2104.07652"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/CompVis/geometry-free-view-synthesis","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":3,"ran_honours":1,"ran_violates":1,"ran_fixture":1,"ran":1,"unverified":2},"by_repo_kind":{"official":{"samples":9,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c3a6b977022957cb","entry":"Normalize","repo":"CompVis/geometry-free-view-synthesis","repo_kind":"official","path":"geofree/modules/diffusionmodules/model.py","file_url":"https://github.com/CompVis/geometry-free-view-synthesis/blob/HEAD/geofree/modules/diffusionmodules/model.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c3a6b977022957cb"}},{"code_sha256_prefix":"613acd72145a95f2","entry":"count_params","repo":"CompVis/geometry-free-view-synthesis","repo_kind":"official","path":"geofree/modules/util.py","file_url":"https://github.com/CompVis/geometry-free-view-synthesis/blob/HEAD/geofree/modules/util.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"613acd72145a95f2"}},{"code_sha256_prefix":"4cb732f513d69dfd","entry":"disabled_train","repo":"CompVis/geometry-free-view-synthesis","repo_kind":"official","path":"geofree/models/transformers/geogpt.py","file_url":"https://github.com/CompVis/geometry-free-view-synthesis/blob/HEAD/geofree/models/transformers/geogpt.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4cb732f513d69dfd"}},{"code_sha256_prefix":"cb49209c125de1b4","entry":"get_timestep_embedding","repo":"CompVis/geometry-free-view-synthesis","repo_kind":"official","path":"geofree/modules/diffusionmodules/model.py","file_url":"https://github.com/CompVis/geometry-free-view-synthesis/blob/HEAD/geofree/modules/diffusionmodules/model.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb49209c125de1b4"}},{"code_sha256_prefix":"3137073275f8c21a","entry":"nonlinearity","repo":"CompVis/geometry-free-view-synthesis","repo_kind":"official","path":"geofree/modules/diffusionmodules/model.py","file_url":"https://github.com/CompVis/geometry-free-view-synthesis/blob/HEAD/geofree/modules/diffusionmodules/model.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3137073275f8c21a"}},{"code_sha256_prefix":"958673f09e3e1cd0","entry":"sample","repo":"CompVis/geometry-free-view-synthesis","repo_kind":"official","path":"geofree/modules/transformer/mingpt.py","file_url":"https://github.com/CompVis/geometry-free-view-synthesis/blob/HEAD/geofree/modules/transformer/mingpt.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"958673f09e3e1cd0"}},{"code_sha256_prefix":"0f0021cdea13e4da","entry":"top_k_logits","repo":"CompVis/geometry-free-view-synthesis","repo_kind":"official","path":"geofree/modules/transformer/mingpt.py","file_url":"https://github.com/CompVis/geometry-free-view-synthesis/blob/HEAD/geofree/modules/transformer/mingpt.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f0021cdea13e4da"}},{"code_sha256_prefix":"c8d9d38b144e401f","entry":"md5_hash","repo":"CompVis/geometry-free-view-synthesis","repo_kind":"official","path":"geofree/util.py","file_url":"https://github.com/CompVis/geometry-free-view-synthesis/blob/HEAD/geofree/util.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c8d9d38b144e401f"}},{"code_sha256_prefix":"dac48a57235412bf","entry":"to_rgb","repo":"CompVis/geometry-free-view-synthesis","repo_kind":"official","path":"geofree/modules/util.py","file_url":"https://github.com/CompVis/geometry-free-view-synthesis/blob/HEAD/geofree/modules/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dac48a57235412bf"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}