Exploration of Reinforcement Learning for Event Camera using Car-like Robots

• URL: http://arxiv.org/abs/2004.00801v1
• Date: Thu, 2 Apr 2020 03:52:03 GMT
• Title: Exploration of Reinforcement Learning for Event Camera using Car-like Robots
• Authors: Riku Arakawa and Shintaro Shiba
• Abstract summary: We demonstrate the first reinforcement-learning application for robots equipped with an event camera. Because of the considerably lower latency of the event camera, it is possible to achieve much faster control of robots.
• Score: 10.66048003460524
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We demonstrate the first reinforcement-learning application for robots equipped with an event camera. Because of the considerably lower latency of the event camera, it is possible to achieve much faster control of robots compared with the existing vision-based reinforcement-learning applications using standard cameras. To handle a stream of events for reinforcement learning, we introduced an image-like feature and demonstrated the feasibility of training an agent in a simulator for two tasks: fast collision avoidance and obstacle tracking. Finally, we set up a robot with an event camera in the real world and then transferred the agent trained in the simulator, resulting in successful fast avoidance of randomly thrown objects. Incorporating event camera into reinforcement learning opens new possibilities for various robotics applications that require swift control, such as autonomous vehicles and drones, through end-to-end learning approaches.

Related papers

• Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulation [65.46610405509338]
  We seek to learn a generalizable goal-conditioned policy that enables zero-shot robot manipulation. Our framework,Track2Act predicts tracks of how points in an image should move in future time-steps based on a goal. We show that this approach of combining scalably learned track prediction with a residual policy enables diverse generalizable robot manipulation.
  arXiv  Detail & Related papers  (2024-05-02T17:56:55Z)
• Robot Learning with Sensorimotor Pre-training [98.7755895548928]
  We present a self-supervised sensorimotor pre-training approach for robotics. Our model, called RPT, is a Transformer that operates on sequences of sensorimotor tokens. We find that sensorimotor pre-training consistently outperforms training from scratch, has favorable scaling properties, and enables transfer across different tasks, environments, and robots.
  arXiv  Detail & Related papers  (2023-06-16T17:58:10Z)
• Scaling Robot Learning with Semantically Imagined Experience [21.361979238427722]
  Recent advances in robot learning have shown promise in enabling robots to perform manipulation tasks. One of the key contributing factors to this progress is the scale of robot data used to train the models. We propose an alternative route and leverage text-to-image foundation models widely used in computer vision and natural language processing.
  arXiv  Detail & Related papers  (2023-02-22T18:47:51Z)
• Learning Reward Functions for Robotic Manipulation by Observing Humans [92.30657414416527]
  We use unlabeled videos of humans solving a wide range of manipulation tasks to learn a task-agnostic reward function for robotic manipulation policies. The learned rewards are based on distances to a goal in an embedding space learned using a time-contrastive objective.
  arXiv  Detail & Related papers  (2022-11-16T16:26:48Z)
• Learning Active Camera for Multi-Object Navigation [94.89618442412247]
  Getting robots to navigate to multiple objects autonomously is essential yet difficult in robot applications. Existing navigation methods mainly focus on fixed cameras and few attempts have been made to navigate with active cameras. In this paper, we consider navigating to multiple objects more efficiently with active cameras.
  arXiv  Detail & Related papers  (2022-10-14T04:17:30Z)
• Learning Semantics-Aware Locomotion Skills from Human Demonstration [35.996425893483796]
  We present a framework that learns semantics-aware locomotion skills from perception for quadrupedal robots. Our framework learns to adjust the speed and gait of the robot based on perceived terrain semantics, and enables the robot to walk over 6km without failure.
  arXiv  Detail & Related papers  (2022-06-27T21:08:03Z)
• Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulation [15.632809977544907]
  Learning to solve precision-based manipulation tasks from visual feedback could drastically reduce the engineering efforts required by traditional robot systems. We propose a setting for robotic manipulation in which the agent receives visual feedback from both a third-person camera and an egocentric camera mounted on the robot's wrist. To fuse visual information from both cameras effectively, we additionally propose to use Transformers with a cross-view attention mechanism.
  arXiv  Detail & Related papers  (2022-01-19T18:39:03Z)
• CNN-based Omnidirectional Object Detection for HermesBot Autonomous Delivery Robot with Preliminary Frame Classification [53.56290185900837]
  We propose an algorithm for optimizing a neural network for object detection using preliminary binary frame classification. An autonomous mobile robot with 6 rolling-shutter cameras on the perimeter providing a 360-degree field of view was used as the experimental setup.
  arXiv  Detail & Related papers  (2021-10-22T15:05:37Z)
• Deep Reinforcement learning for real autonomous mobile robot navigation in indoor environments [0.0]
  We present our proof of concept for autonomous self-learning robot navigation in an unknown environment for a real robot without a map or planner. The input for the robot is only the fused data from a 2D laser scanner and a RGB-D camera as well as the orientation to the goal. The output actions of an Asynchronous Advantage Actor-Critic network (GA3C) are the linear and angular velocities for the robot.
  arXiv  Detail & Related papers  (2020-05-28T09:15:14Z)
• Morphology-Agnostic Visual Robotic Control [76.44045983428701]
  MAVRIC is an approach that works with minimal prior knowledge of the robot's morphology. We demonstrate our method on visually-guided 3D point reaching, trajectory following, and robot-to-robot imitation.
  arXiv  Detail & Related papers  (2019-12-31T15:45:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.