Combined Physics and Event Camera Simulator for Slip Detection

• URL: http://arxiv.org/abs/2503.04838v2
• Date: Mon, 10 Mar 2025 22:49:56 GMT
• Title: Combined Physics and Event Camera Simulator for Slip Detection
• Authors: Thilo Reinold, Suman Ghosh, Guillermo Gallego,
• Abstract summary: This paper presents a simulation pipeline for generating slip data using the described camera-gripper configuration in a robot arm.<n>It provides the ability to alter the setup at any time, simplify the process of repetition and the generation of arbitrarily large data sets.
• Score: 11.309936820480111
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Robot manipulation is a common task in fields like industrial manufacturing. Detecting when objects slip from a robot's grasp is crucial for safe and reliable operation. Event cameras, which register pixel-level brightness changes at high temporal resolution (called ``events''), offer an elegant feature when mounted on a robot's end effector: since they only detect motion relative to their viewpoint, a properly grasped object produces no events, while a slipping object immediately triggers them. To research this feature, representative datasets are essential, both for analytic approaches and for training machine learning models. The majority of current research on slip detection with event-based data is done on real-world scenarios and manual data collection, as well as additional setups for data labeling. This can result in a significant increase in the time required for data collection, a lack of flexibility in scene setups, and a high level of complexity in the repetition of experiments. This paper presents a simulation pipeline for generating slip data using the described camera-gripper configuration in a robot arm, and demonstrates its effectiveness through initial data-driven experiments. The use of a simulator, once it is set up, has the potential to reduce the time spent on data collection, provide the ability to alter the setup at any time, simplify the process of repetition and the generation of arbitrarily large data sets. Two distinct datasets were created and validated through visual inspection and artificial neural networks (ANNs). Visual inspection confirmed photorealistic frame generation and accurate slip modeling, while three ANNs trained on this data achieved high validation accuracy and demonstrated good generalization capabilities on a separate test set, along with initial applicability to real-world data. Project page: https://github.com/tub-rip/event_slip

Related papers

• Synth It Like KITTI: Synthetic Data Generation for Object Detection in Driving Scenarios [3.30184292168618]
  We propose a dataset generation pipeline based on the CARLA simulator for 3D object detection on LiDAR point clouds.<n>We are able to train an object detector on the synthetic data and demonstrate strong generalization capabilities to the KITTI dataset.
  arXiv  Detail & Related papers  (2025-02-20T22:27:42Z)
• eCARLA-scenes: A synthetically generated dataset for event-based optical flow prediction [0.0]
  We introduce eWiz, a comprehensive library for processing event-based data. We present a synthetic event-based datasets and data generation pipelines for optical flow prediction tasks. eCARLA-scenes makes use of the CARLA simulator to simulate self-driving car scenarios.
  arXiv  Detail & Related papers  (2024-12-12T12:02:23Z)
• Synthetica: Large Scale Synthetic Data for Robot Perception [21.415878105900187]
  We present Synthetica, a method for large-scale synthetic data generation for training robust state estimators. This paper focuses on the task of object detection, an important problem which can serve as the front-end for most state estimation problems. We leverage data from a ray-tracing, generating 2.7 million images, to train highly accurate real-time detection transformers. We demonstrate state-of-the-art performance on the task of object detection while having detectors that run at 50-100Hz which is 9 times faster than the prior SOTA.
  arXiv  Detail & Related papers  (2024-10-28T15:50:56Z)
• A New Benchmark: On the Utility of Synthetic Data with Blender for Bare Supervised Learning and Downstream Domain Adaptation [42.2398858786125]
  Deep learning in computer vision has achieved great success with the price of large-scale labeled training data. The uncontrollable data collection process produces non-IID training and test data, where undesired duplication may exist. To circumvent them, an alternative is to generate synthetic data via 3D rendering with domain randomization.
  arXiv  Detail & Related papers  (2023-03-16T09:03:52Z)
• Quantifying the LiDAR Sim-to-Real Domain Shift: A Detailed Investigation Using Object Detectors and Analyzing Point Clouds at Target-Level [1.1999555634662635]
  LiDAR object detection algorithms based on neural networks for autonomous driving require large amounts of data for training, validation, and testing. We show that using simulated data for the training of neural networks leads to a domain shift of training and testing data due to differences in scenes, scenarios, and distributions.
  arXiv  Detail & Related papers  (2023-03-03T12:52:01Z)
• TRoVE: Transforming Road Scene Datasets into Photorealistic Virtual Environments [84.6017003787244]
  This work proposes a synthetic data generation pipeline to address the difficulties and domain-gaps present in simulated datasets. We show that using annotations and visual cues from existing datasets, we can facilitate automated multi-modal data generation.
  arXiv  Detail & Related papers  (2022-08-16T20:46:08Z)
• BeCAPTCHA-Type: Biometric Keystroke Data Generation for Improved Bot Detection [63.447493500066045]
  This work proposes a data driven learning model for the synthesis of keystroke biometric data. The proposed method is compared with two statistical approaches based on Universal and User-dependent models. Our experimental framework considers a dataset with 136 million keystroke events from 168 thousand subjects.
  arXiv  Detail & Related papers  (2022-07-27T09:26:15Z)
• MetaGraspNet: A Large-Scale Benchmark Dataset for Vision-driven Robotic Grasping via Physics-based Metaverse Synthesis [78.26022688167133]
  We present a large-scale benchmark dataset for vision-driven robotic grasping via physics-based metaverse synthesis. The proposed dataset contains 100,000 images and 25 different object types. We also propose a new layout-weighted performance metric alongside the dataset for evaluating object detection and segmentation performance.
  arXiv  Detail & Related papers  (2021-12-29T17:23:24Z)
• Robust Event Classification Using Imperfect Real-world PMU Data [58.26737360525643]
  We study robust event classification using imperfect real-world phasor measurement unit (PMU) data. We develop a novel machine learning framework for training robust event classifiers.
  arXiv  Detail & Related papers  (2021-10-19T17:41:43Z)
• Deep Transformer Networks for Time Series Classification: The NPP Safety Case [59.20947681019466]
  An advanced temporal neural network referred to as the Transformer is used within a supervised learning fashion to model the time-dependent NPP simulation data. The Transformer can learn the characteristics of the sequential data and yield promising performance with approximately 99% classification accuracy on the testing dataset.
  arXiv  Detail & Related papers  (2021-04-09T14:26:25Z)
• Meta-Sim2: Unsupervised Learning of Scene Structure for Synthetic Data Generation [88.04759848307687]
  In Meta-Sim2, we aim to learn the scene structure in addition to parameters, which is a challenging problem due to its discrete nature. We use Reinforcement Learning to train our model, and design a feature space divergence between our synthesized and target images that is key to successful training. We also show that this leads to downstream improvement in the performance of an object detector trained on our generated dataset as opposed to other baseline simulation methods.
  arXiv  Detail & Related papers  (2020-08-20T17:28:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.