Bayesian deep learning of affordances from RGB images

• URL: http://arxiv.org/abs/2109.12845v1
• Date: Mon, 27 Sep 2021 07:39:47 GMT
• Title: Bayesian deep learning of affordances from RGB images
• Authors: Lorenzo Mur-Labadia and Ruben Martinez-Cantin
• Abstract summary: We present a deep learning method to predict the affordances available in the environment directly from RGB images. Based on previous work on socially accepted affordances, our model is based on a multiscale CNN that combines local and global information from the object and the full image. Our results show a marginal better performance of deep ensembles, compared to MC-dropout on the Brier score and the Expected Error.
• Score: 5.939410304994348
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous agents, such as robots or intelligent devices, need to understand how to interact with objects and its environment. Affordances are defined as the relationships between an agent, the objects, and the possible future actions in the environment. In this paper, we present a Bayesian deep learning method to predict the affordances available in the environment directly from RGB images. Based on previous work on socially accepted affordances, our model is based on a multiscale CNN that combines local and global information from the object and the full image. However, previous works assume a deterministic model, but uncertainty quantification is fundamental for robust detection, affordance-based reason, continual learning, etc. Our Bayesian model is able to capture both the aleatoric uncertainty from the scene and the epistemic uncertainty associated with the model and previous learning process. For comparison, we estimate the uncertainty using two state-of-the-art techniques: Monte Carlo dropout and deep ensembles. We also compare different types of CNN encoders for feature extraction. We have performed several experiments on an affordance database on socially acceptable behaviours and we have shown improved performance compared with previous works. Furthermore, the uncertainty estimation is consistent with the the type of objects and scenarios. Our results show a marginal better performance of deep ensembles, compared to MC-dropout on the Brier score and the Expected Calibration Error.

Related papers

• Gaussian Mixture Models for Affordance Learning using Bayesian Networks [50.18477618198277]
  Affordances are fundamental descriptors of relationships between actions, objects and effects. This paper approaches the problem of an embodied agent exploring the world and learning these affordances autonomously from its sensory experiences.
  arXiv  Detail & Related papers  (2024-02-08T22:05:45Z)
• From Reactive to Proactive Volatility Modeling with Hemisphere Neural Networks [0.0]
  We reinvigorate maximum likelihood estimation (MLE) for macroeconomic density forecasting through a novel neural network architecture with dedicated mean and variance hemispheres. Our Hemisphere Neural Network (HNN) provides proactive volatility forecasts based on leading indicators when it can, and reactive volatility based on the magnitude of previous prediction errors when it must.
  arXiv  Detail & Related papers  (2023-11-27T21:37:50Z)
• Single Image Depth Prediction Made Better: A Multivariate Gaussian Take [163.14849753700682]
  We introduce an approach that performs continuous modeling of per-pixel depth. Our method's accuracy (named MG) is among the top on the KITTI depth-prediction benchmark leaderboard.
  arXiv  Detail & Related papers  (2023-03-31T16:01:03Z)
• PRISM: Probabilistic Real-Time Inference in Spatial World Models [52.878769723544615]
  PRISM is a method for real-time filtering in a probabilistic generative model of agent motion and visual perception. The proposed solution runs at 10Hz real-time and is similarly accurate to state-of-the-art SLAM in small to medium-sized indoor environments.
  arXiv  Detail & Related papers  (2022-12-06T13:59:06Z)
• MDN-VO: Estimating Visual Odometry with Confidence [34.8860186009308]
  Visual Odometry (VO) is used in many applications including robotics and autonomous systems. We propose a deep learning-based VO model to estimate 6-DoF poses, as well as a confidence model for these estimates. Our experiments show that the proposed model exceeds state-of-the-art performance in addition to detecting failure cases.
  arXiv  Detail & Related papers  (2021-12-23T19:26:04Z)
• PDC-Net+: Enhanced Probabilistic Dense Correspondence Network [161.76275845530964]
  Enhanced Probabilistic Dense Correspondence Network, PDC-Net+, capable of estimating accurate dense correspondences. We develop an architecture and an enhanced training strategy tailored for robust and generalizable uncertainty prediction. Our approach obtains state-of-the-art results on multiple challenging geometric matching and optical flow datasets.
  arXiv  Detail & Related papers  (2021-09-28T17:56:41Z)
• Towards Robust Object Detection: Bayesian RetinaNet for Homoscedastic Aleatoric Uncertainty Modeling [1.6500749121196985]
  COCO dataset is known for its high level of noise in data labels. We present a series of novel loss functions to address the problem of image object detection at scale.
  arXiv  Detail & Related papers  (2021-08-02T11:03:39Z)
• Contemplating real-world object classification [53.10151901863263]
  We reanalyze the ObjectNet dataset recently proposed by Barbu et al. containing objects in daily life situations. We find that applying deep models to the isolated objects, rather than the entire scene as is done in the original paper, results in around 20-30% performance improvement.
  arXiv  Detail & Related papers  (2021-03-08T23:29:59Z)
• A Novel Regression Loss for Non-Parametric Uncertainty Optimization [7.766663822644739]
  Quantification of uncertainty is one of the most promising approaches to establish safe machine learning. One of the most commonly used approaches so far is Monte Carlo dropout, which is computationally cheap and easy to apply in practice. We propose a new objective, referred to as second-moment loss ( UCI), to address this issue.
  arXiv  Detail & Related papers  (2021-01-07T19:12:06Z)
• Learning Accurate Dense Correspondences and When to Trust Them [161.76275845530964]
  We aim to estimate a dense flow field relating two images, coupled with a robust pixel-wise confidence map. We develop a flexible probabilistic approach that jointly learns the flow prediction and its uncertainty. Our approach obtains state-of-the-art results on challenging geometric matching and optical flow datasets.
  arXiv  Detail & Related papers  (2021-01-05T18:54:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.