The Sound of Water: Inferring Physical Properties from Pouring Liquids

• URL: http://arxiv.org/abs/2411.11222v1
• Date: Mon, 18 Nov 2024 01:19:37 GMT
• Title: The Sound of Water: Inferring Physical Properties from Pouring Liquids
• Authors: Piyush Bagad, Makarand Tapaswi, Cees G. M. Snoek, Andrew Zisserman,
• Abstract summary: We study the connection between audio-visual observations and the underlying physics of pouring liquids. Our objective is to automatically infer physical properties such as the liquid level, the shape and size of the container, the pouring rate and the time to fill.
• Score: 85.30865788636386
• License:
• Abstract: We study the connection between audio-visual observations and the underlying physics of a mundane yet intriguing everyday activity: pouring liquids. Given only the sound of liquid pouring into a container, our objective is to automatically infer physical properties such as the liquid level, the shape and size of the container, the pouring rate and the time to fill. To this end, we: (i) show in theory that these properties can be determined from the fundamental frequency (pitch); (ii) train a pitch detection model with supervision from simulated data and visual data with a physics-inspired objective; (iii) introduce a new large dataset of real pouring videos for a systematic study; (iv) show that the trained model can indeed infer these physical properties for real data; and finally, (v) we demonstrate strong generalization to various container shapes, other datasets, and in-the-wild YouTube videos. Our work presents a keen understanding of a narrow yet rich problem at the intersection of acoustics, physics, and learning. It opens up applications to enhance multisensory perception in robotic pouring.

Related papers

• Latent Intuitive Physics: Learning to Transfer Hidden Physics from A 3D Video [58.043569985784806]
  We introduce latent intuitive physics, a transfer learning framework for physics simulation. It can infer hidden properties of fluids from a single 3D video and simulate the observed fluid in novel scenes. We validate our model in three ways: (i) novel scene simulation with the learned visual-world physics, (ii) future prediction of the observed fluid dynamics, and (iii) supervised particle simulation.
  arXiv  Detail & Related papers  (2024-06-18T16:37:44Z)
• Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusion [35.71595369663293]
  We propose textbfPhysics3D, a novel method for learning various physical properties of 3D objects through a video diffusion model. Our approach involves designing a highly generalizable physical simulation system based on a viscoelastic material model. Experiments demonstrate the effectiveness of our method with both elastic and plastic materials.
  arXiv  Detail & Related papers  (2024-06-06T17:59:47Z)
• Learning Vortex Dynamics for Fluid Inference and Prediction [25.969713036393895]
  We propose a novel machine learning method based on differentiable vortex particles to infer and predict fluid dynamics from a single video. We devise a novel differentiable vortex particle system in conjunction with their learnable, vortex-to-velocity dynamics mapping to effectively capture and represent the complex flow features in a reduced space.
  arXiv  Detail & Related papers  (2023-01-27T02:10:05Z)
• ComPhy: Compositional Physical Reasoning of Objects and Events from Videos [113.2646904729092]
  The compositionality between the visible and hidden properties poses unique challenges for AI models to reason from the physical world. Existing studies on video reasoning mainly focus on visually observable elements such as object appearance, movement, and contact interaction. We propose an oracle neural-symbolic framework named Compositional Physics Learner (CPL), combining visual perception, physical property learning, dynamic prediction, and symbolic execution.
  arXiv  Detail & Related papers  (2022-05-02T17:59:13Z)
• Physics-informed Reinforcement Learning for Perception and Reasoning about Fluids [0.0]
  We propose a physics-informed reinforcement learning strategy for fluid perception and reasoning from observations. We develop a method for the tracking (perception) and analysis (reasoning) of any previously unseen liquid whose free surface is observed with a commodity camera.
  arXiv  Detail & Related papers  (2022-03-11T07:01:23Z)
• NeuroFluid: Fluid Dynamics Grounding with Particle-Driven Neural Radiance Fields [65.07940731309856]
  Deep learning has shown great potential for modeling the physical dynamics of complex particle systems such as fluids. In this paper, we consider a partially observable scenario known as fluid dynamics grounding. We propose a differentiable two-stage network named NeuroFluid. It is shown to reasonably estimate the underlying physics of fluids with different initial shapes, viscosity, and densities.
  arXiv  Detail & Related papers  (2022-03-03T15:13:29Z)
• Visual Grounding of Learned Physical Models [66.04898704928517]
  Humans intuitively recognize objects' physical properties and predict their motion, even when the objects are engaged in complicated interactions. We present a neural model that simultaneously reasons about physics and makes future predictions based on visual and dynamics priors. Experiments show that our model can infer the physical properties within a few observations, which allows the model to quickly adapt to unseen scenarios and make accurate predictions into the future.
  arXiv  Detail & Related papers  (2020-04-28T17:06:38Z)
• Cloth in the Wind: A Case Study of Physical Measurement through Simulation [50.31424339972478]
  We propose to measure latent physical properties for cloth in the wind without ever having seen a real example before. Our solution is an iterative refinement procedure with simulation at its core. The correspondence is measured using an embedding function that maps physically similar examples to nearby points.
  arXiv  Detail & Related papers  (2020-03-09T21:32:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.