{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-sound-of-pixels","title":"The Sound of Pixels","arxiv_id":"1804.03160","date":"2018-04-09","proceeding":"ECCV 2018 9","authors":["Hang Zhao","Chuang Gan","Andrew Rouditchenko","Carl Vondrick","Josh Mcdermott","Antonio Torralba"],"abstract":"We introduce PixelPlayer, a system that, by leveraging large amounts of\nunlabeled videos, learns to locate image regions which produce sounds and\nseparate the input sounds into a set of components that represents the sound\nfrom each pixel. Our approach capitalizes on the natural synchronization of the\nvisual and audio modalities to learn models that jointly parse sounds and\nimages, without requiring additional manual supervision. Experimental results\non a newly collected MUSIC dataset show that our proposed Mix-and-Separate\nframework outperforms several baselines on source separation. Qualitative\nresults suggest our model learns to ground sounds in vision, enabling\napplications such as independently adjusting the volume of sound sources.","url_abs":"http://arxiv.org/abs/1804.03160v4","url_pdf":"http://arxiv.org/pdf/1804.03160v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-sound-of-pixels","repo_url":"https://github.com/binhanle/eem202a-project","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"the-sound-of-pixels","repo_url":"https://github.com/hangzhaomit/Sound-of-Pixels","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1804.03160","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}