Collision Replay: What Does Bumping Into Things Tell You About Scene Geometry?

• URL: http://arxiv.org/abs/2105.01061v1
• Date: Mon, 3 May 2021 17:59:46 GMT
• Title: Collision Replay: What Does Bumping Into Things Tell You About Scene Geometry?
• Authors: Alexander Raistrick, Nilesh Kulkarni, David F. Fouhey
• Abstract summary: We use examples of collisions to provide supervision for observations at a past frame. We use collision replay to train convolutional neural networks to predict a distribution over collision time from new images. We analyze this approach with an agent that has noisy actuation in a photorealistic simulator.
• Score: 87.63134188675717
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: What does bumping into things in a scene tell you about scene geometry? In this paper, we investigate the idea of learning from collisions. At the heart of our approach is the idea of collision replay, where we use examples of a collision to provide supervision for observations at a past frame. We use collision replay to train convolutional neural networks to predict a distribution over collision time from new images. This distribution conveys information about the navigational affordances (e.g., corridors vs open spaces) and, as we show, can be converted into the distance function for the scene geometry. We analyze this approach with an agent that has noisy actuation in a photorealistic simulator.

Related papers

• CabiNet: Scaling Neural Collision Detection for Object Rearrangement with Procedural Scene Generation [54.68738348071891]
  We first generate over 650K cluttered scenes - orders of magnitude more than prior work - in diverse everyday environments. We render synthetic partial point clouds from this data and use it to train our CabiNet model architecture. CabiNet is a collision model that accepts object and scene point clouds, captured from a single-view depth observation.
  arXiv  Detail & Related papers  (2023-04-18T21:09:55Z)
• COPILOT: Human-Environment Collision Prediction and Localization from Egocentric Videos [62.34712951567793]
  The ability to forecast human-environment collisions from egocentric observations is vital to enable collision avoidance in applications such as VR, AR, and wearable assistive robotics. We introduce the challenging problem of predicting collisions in diverse environments from multi-view egocentric videos captured from body-mounted cameras. We propose a transformer-based model called COPILOT to perform collision prediction and localization simultaneously.
  arXiv  Detail & Related papers  (2022-10-04T17:49:23Z)
• Active Learning of Neural Collision Handler for Complex 3D Mesh Deformations [68.0524382279567]
  We present a robust learning algorithm to detect and handle collisions in 3D deforming meshes. Our approach outperforms supervised learning methods and achieves $93.8-98.1%$ accuracy.
  arXiv  Detail & Related papers  (2021-10-08T04:08:31Z)
• Moving SLAM: Fully Unsupervised Deep Learning in Non-Rigid Scenes [85.56602190773684]
  We build on the idea of view synthesis, which uses classical camera geometry to re-render a source image from a different point-of-view. By minimizing the error between the synthetic image and the corresponding real image in a video, the deep network that predicts pose and depth can be trained completely unsupervised.
  arXiv  Detail & Related papers  (2021-05-05T17:08:10Z)
• Predicting and Attending to Damaging Collisions for Placing Everyday Objects in Photo-Realistic Simulations [27.312610846200187]
  We show that a rule-based approach that uses plane detection, to detect free areas, performs poorly. We develop PonNet, which has multimodal attention branches and a self-attention mechanism to predict damaging collisions. Our method can visualize the risk of damaging collisions, which is convenient because it enables the user to understand the risk.
  arXiv  Detail & Related papers  (2021-02-12T13:21:45Z)
• COLLIDE-PRED: Prediction of On-Road Collision From Surveillance Videos [0.0]
  We propose an end-to-end collision prediction system, named as COLLIDE-PRED, to predict collisions in videos. The proposed method is experimentally validated with a number of different videos and proves to be effective in identifying accident in advance.
  arXiv  Detail & Related papers  (2021-01-21T06:45:56Z)
• Object Rearrangement Using Learned Implicit Collision Functions [61.90305371998561]
  We propose a learned collision model that accepts scene and query object point clouds and predicts collisions for 6DOF object poses within the scene. We leverage the learned collision model as part of a model predictive path integral (MPPI) policy in a tabletop rearrangement task. The learned model outperforms both traditional pipelines and learned ablations by 9.8% in accuracy on a dataset of simulated collision queries.
  arXiv  Detail & Related papers  (2020-11-21T05:36:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.