{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dreamsim-learning-new-dimensions-of-human-1","title":"DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data","arxiv_id":"2306.09344","date":"2023-06-15","proceeding":"NeurIPS 2023 11","authors":["Stephanie Fu","Netanel Tamir","Shobhita Sundaram","Lucy Chai","Richard Zhang","Tali Dekel","Phillip Isola"],"abstract":"Current perceptual similarity metrics operate at the level of pixels and patches. These metrics compare images in terms of their low-level colors and textures, but fail to capture mid-level similarities and differences in image layout, object pose, and semantic content. In this paper, we develop a perceptual metric that assesses images holistically. Our first step is to collect a new dataset of human similarity judgments over image pairs that are alike in diverse ways. Critical to this dataset is that judgments are nearly automatic and shared by all observers. To achieve this we use recent text-to-image models to create synthetic pairs that are perturbed along various dimensions. We observe that popular perceptual metrics fall short of explaining our new data, and we introduce a new metric, DreamSim, tuned to better align with human perception. We analyze how our metric is affected by different visual attributes, and find that it focuses heavily on foreground objects and semantic content while also being sensitive to color and layout. Notably, despite being trained on synthetic data, our metric generalizes to real images, giving strong results on retrieval and reconstruction tasks. Furthermore, our metric outperforms both prior learned metrics and recent large vision models on these tasks.","url_abs":"https://arxiv.org/abs/2306.09344v3","url_pdf":"https://arxiv.org/pdf/2306.09344v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dreamsim-learning-new-dimensions-of-human-1","repo_url":"https://github.com/ssundaram21/dreamsim","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[{"method_slug":"align","method_name":"ALIGN"}],"datasets_introduced":[{"slug":"nights","name":"NIGHTS","full_name":"Novel Image Generations with Human-Tested Similarity"}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2306.09344","atlas_url":"https://app.syntology.ai/?focus=2306.09344","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.09344"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ssundaram21/dreamsim","reach":null}],"summary":{"ran_honours":1,"unverified":1},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c64981161de78e53","entry":"normalize_embedding","repo":"ssundaram21/dreamsim","repo_kind":"official","path":"dreamsim/model.py","file_url":"https://github.com/ssundaram21/dreamsim/blob/HEAD/dreamsim/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c64981161de78e53"}},{"code_sha256_prefix":"a4dc9cdf132de932","entry":"download_weights","repo":"ssundaram21/dreamsim","repo_kind":"official","path":"dreamsim/model.py","file_url":"https://github.com/ssundaram21/dreamsim/blob/HEAD/dreamsim/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a4dc9cdf132de932"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}