SEBVS: Synthetic Event-based Visual Servoing for Robot Navigation and Manipulation

• URL: http://arxiv.org/abs/2508.17643v1
• Date: Mon, 25 Aug 2025 04:14:04 GMT
• Title: SEBVS: Synthetic Event-based Visual Servoing for Robot Navigation and Manipulation
• Authors: Krishna Vinod, Prithvi Jai Ramesh, Pavan Kumar B N, Bharatesh Chakravarthi,
• Abstract summary: This work presents an open-source, user-friendly v2e robotics operating system (ROS) package for Gazebo simulation.<n>The package is used to investigate event-based robotic policies for real-time navigation and manipulation.<n>Results highlight the potential of event-driven perception to improve real-time robotic navigation and manipulation.
• Score: 1.983259889617852
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Event cameras offer microsecond latency, high dynamic range, and low power consumption, making them ideal for real-time robotic perception under challenging conditions such as motion blur, occlusion, and illumination changes. However, despite their advantages, synthetic event-based vision remains largely unexplored in mainstream robotics simulators. This lack of simulation setup hinders the evaluation of event-driven approaches for robotic manipulation and navigation tasks. This work presents an open-source, user-friendly v2e robotics operating system (ROS) package for Gazebo simulation that enables seamless event stream generation from RGB camera feeds. The package is used to investigate event-based robotic policies (ERP) for real-time navigation and manipulation. Two representative scenarios are evaluated: (1) object following with a mobile robot and (2) object detection and grasping with a robotic manipulator. Transformer-based ERPs are trained by behavior cloning and compared to RGB-based counterparts under various operating conditions. Experimental results show that event-guided policies consistently deliver competitive advantages. The results highlight the potential of event-driven perception to improve real-time robotic navigation and manipulation, providing a foundation for broader integration of event cameras into robotic policy learning. The GitHub repo for the dataset and code: https://eventbasedvision.github.io/SEBVS/

Related papers

• MiVLA: Towards Generalizable Vision-Language-Action Model with Human-Robot Mutual Imitation Pre-training [102.850162490626]
  We propose MiVLA, a vision-language-action model empowered by human-robot mutual imitation pre-training.<n>We show that MiVLA achieves strong improved generalization capability, outperforming state-of-the-art VLAs.
  arXiv  Detail & Related papers  (2025-12-17T12:59:41Z)
• Robot Control Stack: A Lean Ecosystem for Robot Learning at Scale [11.166320712764465]
  Vision-Language-Action models (VLAs) replace specialized architectures and task-tailored components of expert policies with large-scale data collection and setup-specific fine-tuning.<n>Traditional robotics software frameworks become a bottleneck, while robot simulations offer only limited support for transitioning from and to real-world experiments.<n>We introduce Robot Control Stack (RCS), a lean ecosystem designed from the ground up to support research in robot learning with large-scale generalist policies.
  arXiv  Detail & Related papers  (2025-09-18T13:12:16Z)
• Toward Human-Robot Teaming: Learning Handover Behaviors from 3D Scenes [28.930178662944446]
  We introduce a method for training HRT policies, focusing on human-to-robot handovers, solely from RGB images.<n>We generate robot demonstrations containing image-action pairs captured with a camera mounted on the robot gripper.<n>As a result, the simulated camera pose changes in the reconstructed scene can be directly translated into gripper pose changes.
  arXiv  Detail & Related papers  (2025-08-13T14:47:31Z)
• Learning human-to-robot handovers through 3D scene reconstruction [28.930178662944446]
  We propose the first method for learning supervised-based robot handovers solely from RGB images.<n>We generate robot demonstrations containing image-action pairs captured with a camera mounted on the robot gripper.<n>As a result, the simulated camera pose changes in the reconstructed scene can be directly translated into gripper pose changes.
  arXiv  Detail & Related papers  (2025-07-11T16:26:31Z)
• Taccel: Scaling Up Vision-based Tactile Robotics via High-performance GPU Simulation [50.34179054785646]
  We present Taccel, a high-performance simulation platform that integrates IPC and ABD to model robots, tactile sensors, and objects with both accuracy and unprecedented speed.<n>Taccel provides precise physics simulation and realistic tactile signals while supporting flexible robot-sensor configurations through user-friendly APIs.<n>These capabilities position Taccel as a powerful tool for scaling up tactile robotics research and development.
  arXiv  Detail & Related papers  (2025-04-17T12:57:11Z)
• VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulation [53.63540587160549]
  VidBot is a framework enabling zero-shot robotic manipulation using learned 3D affordance from in-the-wild monocular RGB-only human videos.<n> VidBot paves the way for leveraging everyday human videos to make robot learning more scalable.
  arXiv  Detail & Related papers  (2025-03-10T10:04:58Z)
• Sim-to-Real Transfer for Mobile Robots with Reinforcement Learning: from NVIDIA Isaac Sim to Gazebo and Real ROS 2 Robots [1.2773537446441052]
  This article focuses on demonstrating the applications of Isaac in local planning and obstacle avoidance.<n>We benchmark end-to-end policies with the state-of-the-art Nav2, navigation stack in Robot Operating System (ROS)<n>We also cover the sim-to-real transfer process by demonstrating zero-shot transferability of policies trained in the Isaac simulator to real-world robots.
  arXiv  Detail & Related papers  (2025-01-06T10:26:16Z)
• Polaris: Open-ended Interactive Robotic Manipulation via Syn2Real Visual Grounding and Large Language Models [53.22792173053473]
  We introduce an interactive robotic manipulation framework called Polaris. Polaris integrates perception and interaction by utilizing GPT-4 alongside grounded vision models. We propose a novel Synthetic-to-Real (Syn2Real) pose estimation pipeline.
  arXiv  Detail & Related papers  (2024-08-15T06:40:38Z)
• Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulation [65.46610405509338]
  We seek to learn a generalizable goal-conditioned policy that enables zero-shot robot manipulation. Our framework,Track2Act predicts tracks of how points in an image should move in future time-steps based on a goal. We show that this approach of combining scalably learned track prediction with a residual policy enables diverse generalizable robot manipulation.
  arXiv  Detail & Related papers  (2024-05-02T17:56:55Z)
• RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation [77.41969287400977]
  This paper presents textbfRobotScript, a platform for a deployable robot manipulation pipeline powered by code generation. We also present a benchmark for a code generation benchmark for robot manipulation tasks in free-form natural language. We demonstrate the adaptability of our code generation framework across multiple robot embodiments, including the Franka and UR5 robot arms.
  arXiv  Detail & Related papers  (2024-02-22T15:12:00Z)
• HomeRobot: Open-Vocabulary Mobile Manipulation [107.05702777141178]
  Open-Vocabulary Mobile Manipulation (OVMM) is the problem of picking any object in any unseen environment, and placing it in a commanded location. HomeRobot has two components: a simulation component, which uses a large and diverse curated object set in new, high-quality multi-room home environments; and a real-world component, providing a software stack for the low-cost Hello Robot Stretch.
  arXiv  Detail & Related papers  (2023-06-20T14:30:32Z)
• Neuromorphic Eye-in-Hand Visual Servoing [0.9949801888214528]
  Event cameras give human-like vision capabilities with low latency and wide dynamic range. We present a visual servoing method using an event camera and a switching control strategy to explore, reach and grasp. Experiments prove the effectiveness of the method to track and grasp objects of different shapes without the need for re-tuning.
  arXiv  Detail & Related papers  (2020-04-15T23:57:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.