{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/viewformer-nerf-free-neural-rendering-from","title":"ViewFormer: NeRF-free Neural Rendering from Few Images Using Transformers","arxiv_id":"2203.10157","date":"2022-03-18","proceeding":null,"authors":["Jonáš Kulhánek","Erik Derner","Torsten Sattler","Robert Babuška"],"abstract":"Novel view synthesis is a long-standing problem. In this work, we consider a variant of the problem where we are given only a few context views sparsely covering a scene or an object. The goal is to predict novel viewpoints in the scene, which requires learning priors. The current state of the art is based on Neural Radiance Field (NeRF), and while achieving impressive results, the methods suffer from long training times as they require evaluating millions of 3D point samples via a neural network for each image. We propose a 2D-only method that maps multiple context views and a query pose to a new image in a single pass of a neural network. Our model uses a two-stage architecture consisting of a codebook and a transformer model. The codebook is used to embed individual images into a smaller latent space, and the transformer solves the view synthesis task in this more compact space. To train our model efficiently, we introduce a novel branching attention mechanism that allows us to use the same model not only for neural rendering but also for camera pose estimation. Experimental results on real-world scenes show that our approach is competitive compared to NeRF-based methods while not reasoning explicitly in 3D, and it is faster to train.","url_abs":"https://arxiv.org/abs/2203.10157v2","url_pdf":"https://arxiv.org/pdf/2203.10157v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"viewformer-nerf-free-neural-rendering-from","repo_url":"https://github.com/jkulhanek/viewformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"camera-pose-estimation","task_name":"Camera Pose Estimation"},{"task_slug":"nerf","task_name":"NeRF"},{"task_slug":"neural-rendering","task_name":"Neural Rendering"},{"task_slug":"novel-view-synthesis","task_name":"Novel View Synthesis"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2203.10157","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.10157"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/jkulhanek/viewformer","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":2,"unverified":6},"by_repo_kind":{"official":{"samples":8,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c3a6b977022957cb","entry":"Normalize","repo":"jkulhanek/viewformer","repo_kind":"official","path":"viewformer/models/vqgan_th.py","file_url":"https://github.com/jkulhanek/viewformer/blob/HEAD/viewformer/models/vqgan_th.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c3a6b977022957cb"}},{"code_sha256_prefix":"3137073275f8c21a","entry":"nonlinearity","repo":"jkulhanek/viewformer","repo_kind":"official","path":"viewformer/models/vqgan_th.py","file_url":"https://github.com/jkulhanek/viewformer/blob/HEAD/viewformer/models/vqgan_th.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3137073275f8c21a"}},{"code_sha256_prefix":"da8b4337de3adaef","entry":"angle_absolute_difference","repo":"jkulhanek/viewformer","repo_kind":"official","path":"viewformer/models/utils.py","file_url":"https://github.com/jkulhanek/viewformer/blob/HEAD/viewformer/models/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"da8b4337de3adaef"}},{"code_sha256_prefix":"ece3a4b7a9166849","entry":"angle_squared_difference","repo":"jkulhanek/viewformer","repo_kind":"official","path":"viewformer/models/utils.py","file_url":"https://github.com/jkulhanek/viewformer/blob/HEAD/viewformer/models/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ece3a4b7a9166849"}},{"code_sha256_prefix":"3f98e4ff1d6fb83a","entry":"asdict","repo":"jkulhanek/viewformer","repo_kind":"official","path":"viewformer/models/config.py","file_url":"https://github.com/jkulhanek/viewformer/blob/HEAD/viewformer/models/config.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3f98e4ff1d6fb83a"}},{"code_sha256_prefix":"6add6728d7ad3195","entry":"nonlinearity","repo":"jkulhanek/viewformer","repo_kind":"official","path":"viewformer/models/vqgan.py","file_url":"https://github.com/jkulhanek/viewformer/blob/HEAD/viewformer/models/vqgan.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6add6728d7ad3195"}},{"code_sha256_prefix":"6a6f58f4fdc4300b","entry":"pose_criterion","repo":"jkulhanek/viewformer","repo_kind":"official","path":"viewformer/models/utils.py","file_url":"https://github.com/jkulhanek/viewformer/blob/HEAD/viewformer/models/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6a6f58f4fdc4300b"}},{"code_sha256_prefix":"34a9c5ba83fc96fc","entry":"sparse_softmax_cross_entropy_with_logits","repo":"jkulhanek/viewformer","repo_kind":"official","path":"viewformer/models/migt.py","file_url":"https://github.com/jkulhanek/viewformer/blob/HEAD/viewformer/models/migt.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"34a9c5ba83fc96fc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}