Object Learning and Robust 3D Reconstruction

• URL: http://arxiv.org/abs/2504.17812v1
• Date: Tue, 22 Apr 2025 21:48:31 GMT
• Title: Object Learning and Robust 3D Reconstruction
• Authors: Sara Sabour,
• Abstract summary: We discuss architectural designs and training methods for a neural network to dissect an image into objects of interest without supervision.<n>FlowCapsules uses motion as a cue for the objects of interest in 2D scenarios.<n>We leverage the geometric consistency of scenes in 3D to detect the inconsistent dynamic objects.
• Score: 7.092348056331202
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this thesis we discuss architectural designs and training methods for a neural network to have the ability of dissecting an image into objects of interest without supervision. The main challenge in 2D unsupervised object segmentation is distinguishing between foreground objects of interest and background. FlowCapsules uses motion as a cue for the objects of interest in 2D scenarios. The last part of this thesis focuses on 3D applications where the goal is detecting and removal of the object of interest from the input images. In these tasks, we leverage the geometric consistency of scenes in 3D to detect the inconsistent dynamic objects. Our transient object masks are then used for designing robust optimization kernels to improve 3D modelling in a casual capture setup. One of our goals in this thesis is to show the merits of unsupervised object based approaches in computer vision. Furthermore, we suggest possible directions for defining objects of interest or foreground objects without requiring supervision. Our hope is to motivate and excite the community into further exploring explicit object representations in image understanding tasks.

Related papers

• Towards Flexible 3D Perception: Object-Centric Occupancy Completion Augments 3D Object Detection [54.78470057491049]
  Occupancy has emerged as a promising alternative for 3D scene perception.<n>We introduce object-centric occupancy as a supplement to object bboxes.<n>We show that our occupancy features significantly enhance the detection results of state-of-the-art 3D object detectors.
  arXiv  Detail & Related papers  (2024-12-06T16:12:38Z)
• SUGAR: Pre-training 3D Visual Representations for Robotics [85.55534363501131]
  We introduce a novel 3D pre-training framework for robotics named SUGAR. SUGAR captures semantic, geometric and affordance properties of objects through 3D point clouds. We show that SUGAR's 3D representation outperforms state-of-the-art 2D and 3D representations.
  arXiv  Detail & Related papers  (2024-04-01T21:23:03Z)
• LocaliseBot: Multi-view 3D object localisation with differentiable rendering for robot grasping [9.690844449175948]
  We focus on object pose estimation. Our approach relies on three pieces of information: multiple views of the object, the camera's parameters at those viewpoints, and 3D CAD models of objects. We show that the estimated object pose results in 99.65% grasp accuracy with the ground truth grasp candidates.
  arXiv  Detail & Related papers  (2023-11-14T14:27:53Z)
• Grounding 3D Object Affordance from 2D Interactions in Images [128.6316708679246]
  Grounding 3D object affordance seeks to locate objects' ''action possibilities'' regions in the 3D space. Humans possess the ability to perceive object affordances in the physical world through demonstration images or videos. We devise an Interaction-driven 3D Affordance Grounding Network (IAG), which aligns the region feature of objects from different sources.
  arXiv  Detail & Related papers  (2023-03-18T15:37:35Z)
• 3D Object Aided Self-Supervised Monocular Depth Estimation [5.579605877061333]
  We propose a new method to address dynamic object movements through monocular 3D object detection. Specifically, we first detect 3D objects in the images and build the per-pixel correspondence of the dynamic pixels with the detected object pose. In this way, the depth of every pixel can be learned via a meaningful geometry model.
  arXiv  Detail & Related papers  (2022-12-04T08:52:33Z)
• LaTeRF: Label and Text Driven Object Radiance Fields [8.191404990730236]
  We introduce LaTeRF, a method for extracting an object of interest from a scene given 2D images of the entire scene and known camera poses. To faithfully extract the object from the scene, LaTeRF extends the NeRF formulation with an additional objectness' probability at each 3D point. We demonstrate high-fidelity object extraction on both synthetic and real datasets.
  arXiv  Detail & Related papers  (2022-07-04T17:07:57Z)
• Attentive and Contrastive Learning for Joint Depth and Motion Field Estimation [76.58256020932312]
  Estimating the motion of the camera together with the 3D structure of the scene from a monocular vision system is a complex task. We present a self-supervised learning framework for 3D object motion field estimation from monocular videos.
  arXiv  Detail & Related papers  (2021-10-13T16:45:01Z)
• Object Wake-up: 3-D Object Reconstruction, Animation, and in-situ Rendering from a Single Image [58.69732754597448]
  Given a picture of a chair, could we extract the 3-D shape of the chair, animate its plausible articulations and motions, and render in-situ in its original image space? We devise an automated approach to extract and manipulate articulated objects in single images.
  arXiv  Detail & Related papers  (2021-08-05T16:20:12Z)
• Supervised Training of Dense Object Nets using Optimal Descriptors for Industrial Robotic Applications [57.87136703404356]
  Dense Object Nets (DONs) by Florence, Manuelli and Tedrake introduced dense object descriptors as a novel visual object representation for the robotics community. In this paper we show that given a 3D model of an object, we can generate its descriptor space image, which allows for supervised training of DONs. We compare the training methods on generating 6D grasps for industrial objects and show that our novel supervised training approach improves the pick-and-place performance in industry-relevant tasks.
  arXiv  Detail & Related papers  (2021-02-16T11:40:12Z)
• Learning to Reconstruct and Segment 3D Objects [4.709764624933227]
  We aim to understand scenes and the objects within them by learning general and robust representations using deep neural networks. This thesis makes three core contributions from object-level 3D shape estimation from single or multiple views to scene-level semantic understanding.
  arXiv  Detail & Related papers  (2020-10-19T15:09:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.