AcTExplore: Active Tactile Exploration of Unknown Objects

• URL: http://arxiv.org/abs/2310.08745v3
• Date: Thu, 20 Jun 2024 21:07:32 GMT
• Title: AcTExplore: Active Tactile Exploration of Unknown Objects
• Authors: Amir-Hossein Shahidzadeh, Seong Jong Yoo, Pavan Mantripragada, Chahat Deep Singh, Cornelia Fermüller, Yiannis Aloimonos,
• Abstract summary: We present AcTExplore, an active tactile exploration method driven by reinforcement learning for object reconstruction at scales. Our algorithm incrementally collects tactile data and reconstructs 3D shapes of the objects as well, which can serve as a representation for higher-level downstream tasks. Our method achieves an average of 95.97% IoU coverage on unseen YCB objects while just being trained on primitive shapes.
• Score: 17.755567328263847
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tactile exploration plays a crucial role in understanding object structures for fundamental robotics tasks such as grasping and manipulation. However, efficiently exploring such objects using tactile sensors is challenging, primarily due to the large-scale unknown environments and limited sensing coverage of these sensors. To this end, we present AcTExplore, an active tactile exploration method driven by reinforcement learning for object reconstruction at scales that automatically explores the object surfaces in a limited number of steps. Through sufficient exploration, our algorithm incrementally collects tactile data and reconstructs 3D shapes of the objects as well, which can serve as a representation for higher-level downstream tasks. Our method achieves an average of 95.97% IoU coverage on unseen YCB objects while just being trained on primitive shapes. Project Webpage: https://prg.cs.umd.edu/AcTExplore

Related papers

• Detecting Every Object from Events [24.58024539462497]
  We propose Detecting Every Object in Events (DEOE), an approach tailored for achieving high-speed, class-agnostic open-world object detection in event-based vision. Our code is available at https://github.com/Hatins/DEOE.
  arXiv  Detail & Related papers  (2024-04-08T08:20:53Z)
• RTMDet: An Empirical Study of Designing Real-Time Object Detectors [13.09100888887757]
  We develop an efficient real-time object detector that exceeds the YOLO series and is easily for many object recognition tasks. Together with better training techniques, the resulting object detector achieves, named RTMDet, 52.8% AP on COCO with 300+ FPS on an NVIDIA 3090 GPU. We hope the experimental results can provide new insights into designing versatile real-time object detectors for many object recognition tasks.
  arXiv  Detail & Related papers  (2022-12-14T18:50:20Z)
• Object Manipulation via Visual Target Localization [64.05939029132394]
  Training agents to manipulate objects, poses many challenges. We propose an approach that explores the environment in search for target objects, computes their 3D coordinates once they are located, and then continues to estimate their 3D locations even when the objects are not visible. Our evaluations show a massive 3x improvement in success rate over a model that has access to the same sensory suite.
  arXiv  Detail & Related papers  (2022-03-15T17:59:01Z)
• TANDEM: Learning Joint Exploration and Decision Making with Tactile Sensors [15.418884994244996]
  We focus on the process of guiding tactile exploration, and its interplay with task-related decision making. We propose TANDEM, an architecture to learn efficient exploration strategies in conjunction with decision making. We demonstrate this method on a tactile object recognition task, where a robot equipped with a touch sensor must explore and identify an object from a known set based on tactile feedback alone.
  arXiv  Detail & Related papers  (2022-03-01T23:55:09Z)
• ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representations [52.226947570070784]
  We present Object, a dataset of 100 objects that addresses both challenges with two key innovations. First, Object encodes the visual, auditory, and tactile sensory data for all objects, enabling a number of multisensory object recognition tasks. Second, Object employs a uniform, object-centric simulations, and implicit representation for each object's visual textures, tactile readings, and tactile readings, making the dataset flexible to use and easy to share.
  arXiv  Detail & Related papers  (2021-09-16T14:00:59Z)
• Active 3D Shape Reconstruction from Vision and Touch [66.08432412497443]
  Humans build 3D understandings of the world through active object exploration, using jointly their senses of vision and touch. In 3D shape reconstruction, most recent progress has relied on static datasets of limited sensory data such as RGB images, depth maps or haptic readings. We introduce a system composed of: 1) a haptic simulator leveraging high spatial resolution vision-based tactile sensors for active touching of 3D objects; 2) a mesh-based 3D shape reconstruction model that relies on tactile or visuotactile priors to guide the shape exploration; and 3) a set of data-driven solutions with either tactile or visuo
  arXiv  Detail & Related papers  (2021-07-20T15:56:52Z)
• Analysis of voxel-based 3D object detection methods efficiency for real-time embedded systems [93.73198973454944]
  Two popular voxel-based 3D object detection methods are studied in this paper. Our experiments show that these methods mostly fail to detect distant small objects due to the sparsity of the input point clouds at large distances. Our findings suggest that a considerable part of the computations of existing methods is focused on locations of the scene that do not contribute with successful detection.
  arXiv  Detail & Related papers  (2021-05-21T12:40:59Z)
• Energy Drain of the Object Detection Processing Pipeline for Mobile Devices: Analysis and Implications [77.00418462388525]
  This paper presents the first detailed experimental study of a mobile augmented reality (AR) client's energy consumption and the detection latency of executing Convolutional Neural Networks (CNN) based object detection. Our detailed measurements refine the energy analysis of mobile AR clients and reveal several interesting perspectives regarding the energy consumption of executing CNN-based object detection.
  arXiv  Detail & Related papers  (2020-11-26T00:32:07Z)
• Progressive Object Transfer Detection [84.48927705173494]
  We propose a novel Progressive Object Transfer Detection (POTD) framework. First, POTD can leverage various object supervision of different domains effectively into a progressive detection procedure. Second, POTD consists of two delicate transfer stages, i.e., Low-Shot Transfer Detection (LSTD), and Weakly-Supervised Transfer Detection (WSTD)
  arXiv  Detail & Related papers  (2020-02-12T00:16:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.