Visual-tactile sensing for Real-time liquid Volume Estimation in Grasping

• URL: http://arxiv.org/abs/2202.11503v1
• Date: Wed, 23 Feb 2022 13:38:31 GMT
• Title: Visual-tactile sensing for Real-time liquid Volume Estimation in Grasping
• Authors: Fan Zhu, Ruixing Jia, Lei Yang, Youcan Yan, Zheng Wang, Jia Pan, Wenping Wang
• Abstract summary: We propose a visuo-tactile model for realtime estimation of the liquid inside a deformable container. We fuse two sensory modalities, i.e., the raw visual inputs from the RGB camera and the tactile cues from our specific tactile sensor. The robotic system is well controlled and adjusted based on the estimation model in real time.
• Score: 58.50342759993186
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a deep visuo-tactile model for realtime estimation of the liquid inside a deformable container in a proprioceptive way.We fuse two sensory modalities, i.e., the raw visual inputs from the RGB camera and the tactile cues from our specific tactile sensor without any extra sensor calibrations.The robotic system is well controlled and adjusted based on the estimation model in real time. The main contributions and novelties of our work are listed as follows: 1) Explore a proprioceptive way for liquid volume estimation by developing an end-to-end predictive model with multi-modal convolutional networks, which achieve a high precision with an error of around 2 ml in the experimental validation. 2) Propose a multi-task learning architecture which comprehensively considers the losses from both classification and regression tasks, and comparatively evaluate the performance of each variant on the collected data and actual robotic platform. 3) Utilize the proprioceptive robotic system to accurately serve and control the requested volume of liquid, which is continuously flowing into a deformable container in real time. 4) Adaptively adjust the grasping plan to achieve more stable grasping and manipulation according to the real-time liquid volume prediction.

Related papers

• 4D Contrastive Superflows are Dense 3D Representation Learners [62.433137130087445]
  We introduce SuperFlow, a novel framework designed to harness consecutive LiDAR-camera pairs for establishing pretraining objectives. To further boost learning efficiency, we incorporate a plug-and-play view consistency module that enhances alignment of the knowledge distilled from camera views.
  arXiv  Detail & Related papers  (2024-07-08T17:59:54Z)
• LiqD: A Dynamic Liquid Level Detection Model under Tricky Small Containers [5.361320134021586]
  This paper proposes a container dynamic liquid level detection model based on U2-Net. A large number of experimental results show that the proposed model can effectively detect the dynamic liquid level changes of the liquid in the container.
  arXiv  Detail & Related papers  (2024-03-13T05:53:25Z)
• Inpainting Computational Fluid Dynamics with Deep Learning [8.397730500554047]
  An effective fluid data completion method reduces the required number of sensors in a fluid dynamics experiment. The ill-posed nature of the fluid data completion problem makes it prohibitively difficult to obtain a theoretical solution. We employ the vector quantization technique to map both complete and incomplete fluid data spaces onto discrete-valued lower-dimensional representations.
  arXiv  Detail & Related papers  (2024-02-27T03:44:55Z)
• Fluid Viscosity Prediction Leveraging Computer Vision and Robot Interaction [9.312155153982982]
  This work explores the feasibility of predicting fluid viscosity by analyzing fluid oscillations captured in video data. The pipeline employs a 3D convolutional autoencoder pretrained in a self-supervised manner to extract and learn features from semantic segmentation masks of oscillating fluids. When the latent representations generated by the pretrained autoencoder are used for classification, the system achieves a 97.1% accuracy across a total of 4,140 test datapoints.
  arXiv  Detail & Related papers  (2023-08-04T21:23:34Z)
• StreamYOLO: Real-time Object Detection for Streaming Perception [84.2559631820007]
  We endow the models with the capacity of predicting the future, significantly improving the results for streaming perception. We consider multiple velocities driving scene and propose Velocity-awared streaming AP (VsAP) to jointly evaluate the accuracy. Our simple method achieves the state-of-the-art performance on Argoverse-HD dataset and improves the sAP and VsAP by 4.7% and 8.2% respectively.
  arXiv  Detail & Related papers  (2022-07-21T12:03:02Z)
• Physics-informed machine learning with differentiable programming for heterogeneous underground reservoir pressure management [64.17887333976593]
  Avoiding over-pressurization in subsurface reservoirs is critical for applications like CO2 sequestration and wastewater injection. Managing the pressures by controlling injection/extraction are challenging because of complex heterogeneity in the subsurface. We use differentiable programming with a full-physics model and machine learning to determine the fluid extraction rates that prevent over-pressurization.
  arXiv  Detail & Related papers  (2022-06-21T20:38:13Z)
• Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups [68.8204255655161]
  We present a method to calibrate the parameters of any pair of sensors involving LiDARs, monocular or stereo cameras. The proposed approach can handle devices with very different resolutions and poses, as usually found in vehicle setups.
  arXiv  Detail & Related papers  (2021-01-12T12:02:26Z)
• Fast Modeling and Understanding Fluid Dynamics Systems with Encoder-Decoder Networks [0.0]
  We show that an accurate deep-learning-based proxy model can be taught efficiently by a finite-volume-based simulator. Compared to traditional simulation, the proposed deep learning approach enables much faster forward computation. We quantify the sensitivity of the deep learning model to key physical parameters and hence demonstrate that the inversion problems can be solved with great acceleration.
  arXiv  Detail & Related papers  (2020-06-09T17:14:08Z)
• Faster Depth-Adaptive Transformers [71.20237659479703]
  Depth-adaptive neural networks can dynamically adjust depths according to the hardness of input words. Previous works generally build a halting unit to decide whether the computation should continue or stop at each layer. In this paper, we get rid of the halting unit and estimate the required depths in advance, which yields a faster depth-adaptive model.
  arXiv  Detail & Related papers  (2020-04-27T15:08:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.