{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gpt-4v-ision-is-a-human-aligned-evaluator-for","title":"GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation","arxiv_id":"2401.04092","date":"2024-01-08","proceeding":"CVPR 2024 1","authors":["Tong Wu","Guandao Yang","Zhibing Li","Kai Zhang","Ziwei Liu","Leonidas Guibas","Dahua Lin","Gordon Wetzstein"],"abstract":"Despite recent advances in text-to-3D generative methods, there is a notable absence of reliable evaluation metrics. Existing metrics usually focus on a single criterion each, such as how well the asset aligned with the input text. These metrics lack the flexibility to generalize to different evaluation criteria and might not align well with human preferences. Conducting user preference studies is an alternative that offers both adaptability and human-aligned results. User studies, however, can be very expensive to scale. This paper presents an automatic, versatile, and human-aligned evaluation metric for text-to-3D generative models. To this end, we first develop a prompt generator using GPT-4V to generate evaluating prompts, which serve as input to compare text-to-3D models. We further design a method instructing GPT-4V to compare two 3D assets according to user-defined criteria. Finally, we use these pairwise comparison results to assign these models Elo ratings. Experimental results suggest our metric strongly align with human preference across different evaluation criteria.","url_abs":"https://arxiv.org/abs/2401.04092v2","url_pdf":"https://arxiv.org/pdf/2401.04092v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gpt-4v-ision-is-a-human-aligned-evaluator-for","repo_url":"https://github.com/3DTopia/GPTEval3D","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"3d-generation","task_name":"3D Generation"},{"task_slug":"text-to-3d","task_name":"Text to 3D"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2401.04092","atlas_url":"https://app.syntology.ai/?focus=2401.04092","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.04092"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/3DTopia/GPTEval3D","reach":null}],"summary":{"ran_draft_wrong":2},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"09f44126d5417dba","entry":"call_gpt_4v","repo":"3DTopia/GPTEval3D","repo_kind":"official","path":"utils/gpt4v_utils.py","file_url":"https://github.com/3DTopia/GPTEval3D/blob/HEAD/utils/gpt4v_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"none","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"09f44126d5417dba"}},{"code_sha256_prefix":"f41cb1a19b154297","entry":"encode_image","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"f41cb1a19b154297"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}