{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/enhancing-high-resolution-3d-generation","title":"Enhancing High-Resolution 3D Generation through Pixel-wise Gradient Clipping","arxiv_id":"2310.12474","date":"2023-10-19","proceeding":null,"authors":["Zijie Pan","Jiachen Lu","Xiatian Zhu","Li Zhang"],"abstract":"High-resolution 3D object generation remains a challenging task primarily due to the limited availability of comprehensive annotated training data. Recent advancements have aimed to overcome this constraint by harnessing image generative models, pretrained on extensive curated web datasets, using knowledge transfer techniques like Score Distillation Sampling (SDS). Efficiently addressing the requirements of high-resolution rendering often necessitates the adoption of latent representation-based models, such as the Latent Diffusion Model (LDM). In this framework, a significant challenge arises: To compute gradients for individual image pixels, it is necessary to backpropagate gradients from the designated latent space through the frozen components of the image model, such as the VAE encoder used within LDM. However, this gradient propagation pathway has never been optimized, remaining uncontrolled during training. We find that the unregulated gradients adversely affect the 3D model's capacity in acquiring texture-related information from the image generative model, leading to poor quality appearance synthesis. To address this overarching challenge, we propose an innovative operation termed Pixel-wise Gradient Clipping (PGC) designed for seamless integration into existing 3D generative models, thereby enhancing their synthesis quality. Specifically, we control the magnitude of stochastic gradients by clipping the pixel-wise gradients efficiently, while preserving crucial texture-related gradient directions. Despite this simplicity and minimal extra cost, extensive experiments demonstrate the efficacy of our PGC in enhancing the performance of existing 3D generative models for high-resolution object rendering.","url_abs":"https://arxiv.org/abs/2310.12474v4","url_pdf":"https://arxiv.org/pdf/2310.12474v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"enhancing-high-resolution-3d-generation","repo_url":"https://github.com/fudan-zvg/pgc-3d","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"3d-generation","task_name":"3D Generation"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"gradient-clipping","method_name":"Gradient Clipping"},{"method_slug":"latent-diffusion-model","method_name":"Latent Diffusion Model"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.12474","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.12474"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/fudan-zvg/pgc-3d","reach":null}],"summary":{"ran":3,"unverified":6},"by_repo_kind":{"official":{"samples":9,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"93dcd06df2fc1896","entry":"CrossAttention","repo":"fudan-zvg/pgc-3d","repo_kind":"official","path":"ldm/modules/diffusionmodules/openaimodel.py","file_url":"https://github.com/fudan-zvg/pgc-3d/blob/HEAD/ldm/modules/diffusionmodules/openaimodel.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"93dcd06df2fc1896"}},{"code_sha256_prefix":"db5d8eda94db8878","entry":"QKVAttention","repo":"fudan-zvg/pgc-3d","repo_kind":"official","path":"ldm/modules/diffusionmodules/openaimodel.py","file_url":"https://github.com/fudan-zvg/pgc-3d/blob/HEAD/ldm/modules/diffusionmodules/openaimodel.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"db5d8eda94db8878"}},{"code_sha256_prefix":"d1535f9d0cc44355","entry":"QKVAttentionLegacy","repo":"fudan-zvg/pgc-3d","repo_kind":"official","path":"ldm/modules/diffusionmodules/openaimodel.py","file_url":"https://github.com/fudan-zvg/pgc-3d/blob/HEAD/ldm/modules/diffusionmodules/openaimodel.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d1535f9d0cc44355"}},{"code_sha256_prefix":"d9fb5e776204ff16","entry":"AttentionBlock","repo":"fudan-zvg/pgc-3d","repo_kind":"official","path":"ldm/modules/diffusionmodules/openaimodel.py","file_url":"https://github.com/fudan-zvg/pgc-3d/blob/HEAD/ldm/modules/diffusionmodules/openaimodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d9fb5e776204ff16"}},{"code_sha256_prefix":"ddc045dc2824caf3","entry":"BasicTransformerBlock","repo":"fudan-zvg/pgc-3d","repo_kind":"official","path":"ldm/modules/diffusionmodules/openaimodel.py","file_url":"https://github.com/fudan-zvg/pgc-3d/blob/HEAD/ldm/modules/diffusionmodules/openaimodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ddc045dc2824caf3"}},{"code_sha256_prefix":"dd36a5c7a3b8451c","entry":"ResBlock","repo":"fudan-zvg/pgc-3d","repo_kind":"official","path":"ldm/modules/diffusionmodules/openaimodel.py","file_url":"https://github.com/fudan-zvg/pgc-3d/blob/HEAD/ldm/modules/diffusionmodules/openaimodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dd36a5c7a3b8451c"}},{"code_sha256_prefix":"1967b733828c7686","entry":"SpatialTransformer","repo":"fudan-zvg/pgc-3d","repo_kind":"official","path":"ldm/modules/diffusionmodules/openaimodel.py","file_url":"https://github.com/fudan-zvg/pgc-3d/blob/HEAD/ldm/modules/diffusionmodules/openaimodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1967b733828c7686"}},{"code_sha256_prefix":"52958f7c2c3d2f92","entry":"TimestepEmbedSequential","repo":"fudan-zvg/pgc-3d","repo_kind":"official","path":"ldm/modules/diffusionmodules/openaimodel.py","file_url":"https://github.com/fudan-zvg/pgc-3d/blob/HEAD/ldm/modules/diffusionmodules/openaimodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"52958f7c2c3d2f92"}},{"code_sha256_prefix":"ac7c0c0e36a9cb38","entry":"UNetModel","repo":"fudan-zvg/pgc-3d","repo_kind":"official","path":"ldm/modules/diffusionmodules/openaimodel.py","file_url":"https://github.com/fudan-zvg/pgc-3d/blob/HEAD/ldm/modules/diffusionmodules/openaimodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ac7c0c0e36a9cb38"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}