NeRF in Robotics: A Survey

• URL: http://arxiv.org/abs/2405.01333v1
• Date: Thu, 2 May 2024 14:38:18 GMT
• Title: NeRF in Robotics: A Survey
• Authors: Guangming Wang, Lei Pan, Songyou Peng, Shaohui Liu, Chenfeng Xu, Yanzi Miao, Wei Zhan, Masayoshi Tomizuka, Marc Pollefeys, Hesheng Wang,
• Abstract summary: The recent emergence of neural implicit representations has introduced radical innovation to computer vision and robotics fields. NeRF has sparked a trend because of the huge representational advantages, such as simplified mathematical models, compact environment storage, and continuous scene representations.
• Score: 95.11502610414803
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Meticulous 3D environment representations have been a longstanding goal in computer vision and robotics fields. The recent emergence of neural implicit representations has introduced radical innovation to this field as implicit representations enable numerous capabilities. Among these, the Neural Radiance Field (NeRF) has sparked a trend because of the huge representational advantages, such as simplified mathematical models, compact environment storage, and continuous scene representations. Apart from computer vision, NeRF has also shown tremendous potential in the field of robotics. Thus, we create this survey to provide a comprehensive understanding of NeRF in the field of robotics. By exploring the advantages and limitations of NeRF, as well as its current applications and future potential, we hope to shed light on this promising area of research. Our survey is divided into two main sections: \textit{The Application of NeRF in Robotics} and \textit{The Advance of NeRF in Robotics}, from the perspective of how NeRF enters the field of robotics. In the first section, we introduce and analyze some works that have been or could be used in the field of robotics from the perception and interaction perspectives. In the second section, we show some works related to improving NeRF's own properties, which are essential for deploying NeRF in the field of robotics. In the discussion section of the review, we summarize the existing challenges and provide some valuable future research directions for reference.

Related papers

• Neural Fields in Robotics: A Survey [39.93473561102639]
  Neural Fields have emerged as a transformative approach for 3D scene representation in computer vision and robotics. This survey explores their applications in robotics, emphasizing their potential to enhance perception, planning, and control. Their compactness, memory efficiency, and differentiability, along with seamless integration with foundation and generative models, make them ideal for real-time applications.
  arXiv  Detail & Related papers  (2024-10-26T16:26:41Z)
• Neural Radiance Field-based Visual Rendering: A Comprehensive Review [0.6047429555885261]
  In recent years, Neural Radiance Fields (NeRF) has made remarkable progress in the field of computer vision and graphics. NeRF has caused a continuous research boom in the academic community. This review provides an in-depth analysis of the research literature on NeRF within the past two years.
  arXiv  Detail & Related papers  (2024-03-31T15:18:38Z)
• 3D Visibility-aware Generalizable Neural Radiance Fields for Interacting Hands [51.305421495638434]
  Neural radiance fields (NeRFs) are promising 3D representations for scenes, objects, and humans. This paper proposes a generalizable visibility-aware NeRF framework for interacting hands. Experiments on the Interhand2.6M dataset demonstrate that our proposed VA-NeRF outperforms conventional NeRFs significantly.
  arXiv  Detail & Related papers  (2024-01-02T00:42:06Z)
• DReg-NeRF: Deep Registration for Neural Radiance Fields [66.69049158826677]
  We propose DReg-NeRF to solve the NeRF registration problem on object-centric annotated scenes without human intervention. Our proposed method beats the SOTA point cloud registration methods by a large margin.
  arXiv  Detail & Related papers  (2023-08-18T08:37:49Z)
• BeyondPixels: A Comprehensive Review of the Evolution of Neural Radiance Fields [1.1531932979578041]
  NeRF, short for Neural Radiance Fields, is a recent innovation that uses AI algorithms to create 3D objects from 2D images. This survey reviews recent advances in NeRF and categorizes them according to their architectural designs.
  arXiv  Detail & Related papers  (2023-06-05T16:10:21Z)
• Neural Radiance Fields: Past, Present, and Future [0.0]
  An attempt made by Mildenhall et al in their paper about NeRFs led to a boom in Computer Graphics, Robotics, Computer Vision, and the possible scope of High-Resolution Low Storage Augmented Reality and Virtual Reality-based 3D models have gained traction from res with more than 1000 preprints related to NeRFs published. This survey provides the history of rendering, Implicit Learning, and NeRFs, the progression of research on NeRFs, and the potential applications and implications of NeRFs in today's world.
  arXiv  Detail & Related papers  (2023-04-20T02:17:08Z)
• NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review [19.67372661944804]
  Neural Radiance Field (NeRF) has recently become a significant development in the field of Computer Vision. NeRF models have found diverse applications in robotics, urban mapping, autonomous navigation, virtual reality/augmented reality, and more.
  arXiv  Detail & Related papers  (2022-10-01T21:35:11Z)
• NeRF-Loc: Transformer-Based Object Localization Within Neural Radiance Fields [62.89785701659139]
  We propose a transformer-based framework, NeRF-Loc, to extract 3D bounding boxes of objects in NeRF scenes. NeRF-Loc takes a pre-trained NeRF model and camera view as input and produces labeled, oriented 3D bounding boxes of objects as output.
  arXiv  Detail & Related papers  (2022-09-24T18:34:22Z)
• NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields [54.27264716713327]
  We show that a Neural Radiance Fields (NeRF) representation of a scene can be used to train dense object descriptors. We use an optimized NeRF to extract dense correspondences between multiple views of an object, and then use these correspondences as training data for learning a view-invariant representation of the object. Dense correspondence models supervised with our method significantly outperform off-the-shelf learned descriptors by 106%.
  arXiv  Detail & Related papers  (2022-03-03T18:49:57Z)
• Future Frame Prediction for Robot-assisted Surgery [57.18185972461453]
  We propose a ternary prior guided variational autoencoder (TPG-VAE) model for future frame prediction in robotic surgical video sequences. Besides content distribution, our model learns motion distribution, which is novel to handle the small movements of surgical tools.
  arXiv  Detail & Related papers  (2021-03-18T15:12:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.