{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-sound-of-water-inferring-physical","title":"The Sound of Water: Inferring Physical Properties from Pouring Liquids","arxiv_id":"2411.11222","date":"2024-11-18","proceeding":null,"authors":["Piyush Bagad","Makarand Tapaswi","Cees G. M. Snoek","Andrew Zisserman"],"abstract":"We study the connection between audio-visual observations and the underlying physics of a mundane yet intriguing everyday activity: pouring liquids. Given only the sound of liquid pouring into a container, our objective is to automatically infer physical properties such as the liquid level, the shape and size of the container, the pouring rate and the time to fill. To this end, we: (i) show in theory that these properties can be determined from the fundamental frequency (pitch); (ii) train a pitch detection model with supervision from simulated data and visual data with a physics-inspired objective; (iii) introduce a new large dataset of real pouring videos for a systematic study; (iv) show that the trained model can indeed infer these physical properties for real data; and finally, (v) we demonstrate strong generalization to various container shapes, other datasets, and in-the-wild YouTube videos. Our work presents a keen understanding of a narrow yet rich problem at the intersection of acoustics, physics, and learning. It opens up applications to enhance multisensory perception in robotic pouring.","url_abs":"https://arxiv.org/abs/2411.11222v2","url_pdf":"https://arxiv.org/pdf/2411.11222v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-sound-of-water-inferring-physical","repo_url":"https://github.com/bpiyush/SoundOfWater","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"physical-attribute-prediction","task_name":"Physical Attribute Prediction"}],"methods":[],"datasets_introduced":[{"slug":"sound-of-water-50","name":"Sound of Water 50","full_name":"Sound of Water 50"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/physical-attribute-prediction-on-sound-of","task":"Physical Attribute Prediction","dataset":"Sound of Water 50","model":"PouringNet","rank_in_archive_order":1,"of":1,"metrics":{"Mean Squared Error":"0.60"},"uses_additional_data":true}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}