{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/palette-image-to-image-diffusion-models-1","title":"Palette: Image-to-Image Diffusion Models","arxiv_id":"2111.05826","date":"2021-11-10","proceeding":null,"authors":["Chitwan Saharia","William Chan","Huiwen Chang","Chris A. Lee","Jonathan Ho","Tim Salimans","David J. Fleet","Mohammad Norouzi"],"abstract":"This paper develops a unified framework for image-to-image translation based on conditional diffusion models and evaluates this framework on four challenging image-to-image translation tasks, namely colorization, inpainting, uncropping, and JPEG restoration. Our simple implementation of image-to-image diffusion models outperforms strong GAN and regression baselines on all tasks, without task-specific hyper-parameter tuning, architecture customization, or any auxiliary loss or sophisticated new techniques needed. We uncover the impact of an L2 vs. L1 loss in the denoising diffusion objective on sample diversity, and demonstrate the importance of self-attention in the neural architecture through empirical studies. Importantly, we advocate a unified evaluation protocol based on ImageNet, with human evaluation and sample quality scores (FID, Inception Score, Classification Accuracy of a pre-trained ResNet-50, and Perceptual Distance against original images). We expect this standardized evaluation protocol to play a role in advancing image-to-image translation research. Finally, we show that a generalist, multi-task diffusion model performs as well or better than task-specific specialist counterparts. Check out https://diffusion-palette.github.io for an overview of the results.","url_abs":"https://arxiv.org/abs/2111.05826v2","url_pdf":"https://arxiv.org/pdf/2111.05826v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"palette-image-to-image-diffusion-models-1","repo_url":"https://github.com/Janspiry/Palette-Image-to-Image-Diffusion-Models","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"palette-image-to-image-diffusion-models-1","repo_url":"https://github.com/LouisRouss/Diffusion-Based-Model-for-Colorization","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"palette-image-to-image-diffusion-models-1","repo_url":"https://github.com/crosszamirski/guided-i2i","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"palette-image-to-image-diffusion-models-1","repo_url":"https://github.com/kylelo/roofdiffusion","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"palette-image-to-image-diffusion-models-1","repo_url":"https://github.com/omerb01/puq","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"colorization","task_name":"Colorization"},{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"image-to-image-translation","task_name":"Image-to-Image Translation"},{"task_slug":"jpeg-decompression","task_name":"JPEG Decompression"},{"task_slug":"perceptual-distance","task_name":"Perceptual Distance"},{"task_slug":"translation","task_name":"Translation"},{"task_slug":"uncropping","task_name":"Uncropping"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/colorization-on-imagenet-ctest10k","task":"Colorization","dataset":"ImageNet ctest10k","model":"Palette","rank_in_archive_order":1,"of":1,"metrics":{"FID":"3.4"},"uses_additional_data":false},{"leaderboard":"/sota/colorization-on-imagenet-val","task":"Colorization","dataset":"ImageNet val","model":"Palette","rank_in_archive_order":1,"of":4,"metrics":{"FID-5K":"15.78"},"uses_additional_data":false},{"leaderboard":"/sota/image-inpainting-on-places2-val","task":"Image Inpainting","dataset":"Places2 val","model":"Palatte (20-30% free form)","rank_in_archive_order":1,"of":7,"metrics":{"FID":"11.7","PD":"35.0"},"uses_additional_data":false},{"leaderboard":"/sota/image-inpainting-on-places2-val","task":"Image Inpainting","dataset":"Places2 val","model":"Palette (128×128\ncenter mask)","rank_in_archive_order":2,"of":7,"metrics":{"FID":"11.9","PD":"57.3"},"uses_additional_data":false},{"leaderboard":"/sota/jpeg-decompression-on-imagenet","task":"JPEG Decompression","dataset":"ImageNet","model":"Palette (QF: 20)","rank_in_archive_order":1,"of":6,"metrics":{"CA":"73.5","FID-5K":"4.3","IS":"208.7","PD":"37.1"},"uses_additional_data":false},{"leaderboard":"/sota/jpeg-decompression-on-imagenet","task":"JPEG Decompression","dataset":"ImageNet","model":"Palette (QF: 10)","rank_in_archive_order":2,"of":6,"metrics":{"CA":"70.7","FID-5K":"5.4","IS":"180.5","PD":"58.3"},"uses_additional_data":false},{"leaderboard":"/sota/jpeg-decompression-on-imagenet","task":"JPEG Decompression","dataset":"ImageNet","model":"Palette (QF: 5)","rank_in_archive_order":3,"of":6,"metrics":{"CA":"64.2","FID-5K":"8.3","IS":"133.6","PD":"95.5"},"uses_additional_data":false},{"leaderboard":"/sota/jpeg-decompression-on-imagenet","task":"JPEG Decompression","dataset":"ImageNet","model":"Regression (QF: 20)","rank_in_archive_order":4,"of":6,"metrics":{"CA":"69.7","FID-5K":"11.5","IS":"158.7","PD":"65.4"},"uses_additional_data":false},{"leaderboard":"/sota/jpeg-decompression-on-imagenet","task":"JPEG Decompression","dataset":"ImageNet","model":"Regression (QF: 10)","rank_in_archive_order":5,"of":6,"metrics":{"CA":"63.5","FID-5K":"18.0","IS":"117.2","PD":"102.2"},"uses_additional_data":false},{"leaderboard":"/sota/jpeg-decompression-on-imagenet","task":"JPEG Decompression","dataset":"ImageNet","model":"Regression (QF: 5)","rank_in_archive_order":6,"of":6,"metrics":{"CA":"52.8","FID-5K":"29.0","IS":"73.9","PD":"155.4"},"uses_additional_data":false},{"leaderboard":"/sota/uncropping-on-places2-val","task":"Uncropping","dataset":"Places2 val","model":"Palette","rank_in_archive_order":1,"of":2,"metrics":{"FID":"3.53","Fool rate":"39.9","PD":"103.3"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2111.05826","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2111.05826"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/omerb01/puq","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/crosszamirski/guided-i2i","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/LouisRouss/Diffusion-Based-Model-for-Colorization","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Janspiry/Palette-Image-to-Image-Diffusion-Models","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kylelo/roofdiffusion","reach":null}],"summary":{"ran_honours":1,"ran_fixture":1,"unverified":1},"by_repo_kind":{"listed":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"ec2d227dfc59af9d","entry":"IoU","repo":"kylelo/roofdiffusion","repo_kind":"listed","path":"data/util/roof_metric.py","file_url":"https://github.com/kylelo/roofdiffusion/blob/HEAD/data/util/roof_metric.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ec2d227dfc59af9d"}},{"code_sha256_prefix":"c9f8bc1be2cb6a09","entry":"calculate_metrics","repo":"kylelo/roofdiffusion","repo_kind":"listed","path":"data/util/roof_metric.py","file_url":"https://github.com/kylelo/roofdiffusion/blob/HEAD/data/util/roof_metric.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c9f8bc1be2cb6a09"}},{"code_sha256_prefix":"09c8479d9a5b3e06","entry":"extract","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"09c8479d9a5b3e06"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}