{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/neural-nearest-neighbors-networks","title":"Neural Nearest Neighbors Networks","arxiv_id":"1810.12575","date":"2018-10-30","proceeding":"NeurIPS 2018 12","authors":["Tobias Plötz","Stefan Roth"],"abstract":"Non-local methods exploiting the self-similarity of natural signals have been\nwell studied, for example in image analysis and restoration. Existing\napproaches, however, rely on k-nearest neighbors (KNN) matching in a fixed\nfeature space. The main hurdle in optimizing this feature space w.r.t.\napplication performance is the non-differentiability of the KNN selection rule.\nTo overcome this, we propose a continuous deterministic relaxation of KNN\nselection that maintains differentiability w.r.t. pairwise distances, but\nretains the original KNN as the limit of a temperature parameter approaching\nzero. To exploit our relaxation, we propose the neural nearest neighbors block\n(N3 block), a novel non-local processing layer that leverages the principle of\nself-similarity and can be used as building block in modern neural network\narchitectures. We show its effectiveness for the set reasoning task of\ncorrespondence classification as well as for image restoration, including image\ndenoising and single image super-resolution, where we outperform strong\nconvolutional neural network (CNN) baselines and recent non-local models that\nrely on KNN selection in hand-chosen features spaces.","url_abs":"http://arxiv.org/abs/1810.12575v1","url_pdf":"http://arxiv.org/pdf/1810.12575v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"neural-nearest-neighbors-networks","repo_url":"https://github.com/visinf/n3net","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"neural-nearest-neighbors-networks","repo_url":"https://github.com/Build-Week-Spotify-Song-Suggester-5/Data-Science","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"image-denoising","task_name":"Image Denoising"},{"task_slug":"image-restoration","task_name":"Image Restoration"},{"task_slug":"image-super-resolution","task_name":"Image Super-Resolution"},{"task_slug":"super-resolution","task_name":"Super-Resolution"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/grayscale-image-denoising-on-bsd68-sigma25","task":"Grayscale Image Denoising","dataset":"BSD68 sigma25","model":"N3Net","rank_in_archive_order":8,"of":16,"metrics":{"PSNR":"29.3"},"uses_additional_data":false},{"leaderboard":"/sota/grayscale-image-denoising-on-bsd68-sigma50","task":"Grayscale Image Denoising","dataset":"BSD68 sigma50","model":"N3Net","rank_in_archive_order":9,"of":15,"metrics":{"PSNR":"26.39"},"uses_additional_data":false},{"leaderboard":"/sota/grayscale-image-denoising-on-bsd68-sigma70","task":"Grayscale Image Denoising","dataset":"BSD68 sigma70","model":"N3Net","rank_in_archive_order":1,"of":3,"metrics":{"PSNR":"25.14"},"uses_additional_data":false},{"leaderboard":"/sota/grayscale-image-denoising-on-set12-sigma25","task":"Grayscale Image Denoising","dataset":"Set12 sigma25","model":"N3Net","rank_in_archive_order":5,"of":6,"metrics":{"PSNR":"30.55"},"uses_additional_data":false},{"leaderboard":"/sota/grayscale-image-denoising-on-set12-sigma50","task":"Grayscale Image Denoising","dataset":"Set12 sigma50","model":"N3Net","rank_in_archive_order":6,"of":8,"metrics":{"PSNR":"27.43"},"uses_additional_data":false},{"leaderboard":"/sota/grayscale-image-denoising-on-set12-sigma70","task":"Grayscale Image Denoising","dataset":"Set12 sigma70","model":"N3Net","rank_in_archive_order":1,"of":1,"metrics":{"PSNR":"25.9"},"uses_additional_data":false},{"leaderboard":"/sota/grayscale-image-denoising-on-urban100-sigma25","task":"Grayscale Image Denoising","dataset":"Urban100 sigma25","model":"N3Net","rank_in_archive_order":9,"of":10,"metrics":{"PSNR":"30.19","SSIM":"0.892"},"uses_additional_data":false},{"leaderboard":"/sota/grayscale-image-denoising-on-urban100-sigma50","task":"Grayscale Image Denoising","dataset":"Urban100 sigma50","model":"N3Net","rank_in_archive_order":10,"of":10,"metrics":{"PSNR":"26.82"},"uses_additional_data":false},{"leaderboard":"/sota/grayscale-image-denoising-on-urban100-sigma70","task":"Grayscale Image Denoising","dataset":"Urban100 sigma70","model":"N3Net","rank_in_archive_order":2,"of":2,"metrics":{"PSNR":"25.15"},"uses_additional_data":false},{"leaderboard":"/sota/image-super-resolution-on-set5-2x-upscaling","task":"Image Super-Resolution","dataset":"Set5 - 2x upscaling","model":"N3Net","rank_in_archive_order":35,"of":41,"metrics":{"PSNR":"37.57"},"uses_additional_data":false},{"leaderboard":"/sota/image-super-resolution-on-set5-3x-upscaling","task":"Image Super-Resolution","dataset":"Set5 - 3x upscaling","model":"N3Net","rank_in_archive_order":27,"of":32,"metrics":{"PSNR":"33.84"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1810.12575","atlas_url":"https://app.syntology.ai/?focus=1810.12575","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}