Ego-motion and Surrounding Vehicle State Estimation Using a Monocular Camera

• URL: http://arxiv.org/abs/2005.01632v3
• Date: Wed, 6 May 2020 00:52:46 GMT
• Title: Ego-motion and Surrounding Vehicle State Estimation Using a Monocular Camera
• Authors: Jun Hayakawa, Behzad Dariush
• Abstract summary: We propose a novel machine learning method to estimate ego-motion and surrounding vehicle state using a single monocular camera. Our approach is based on a combination of three deep neural networks to estimate the 3D vehicle bounding box, depth, and optical flow from a sequence of images.
• Score: 11.29865843123467
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Understanding ego-motion and surrounding vehicle state is essential to enable automated driving and advanced driving assistance technologies. Typical approaches to solve this problem use fusion of multiple sensors such as LiDAR, camera, and radar to recognize surrounding vehicle state, including position, velocity, and orientation. Such sensing modalities are overly complex and costly for production of personal use vehicles. In this paper, we propose a novel machine learning method to estimate ego-motion and surrounding vehicle state using a single monocular camera. Our approach is based on a combination of three deep neural networks to estimate the 3D vehicle bounding box, depth, and optical flow from a sequence of images. The main contribution of this paper is a new framework and algorithm that integrates these three networks in order to estimate the ego-motion and surrounding vehicle state. To realize more accurate 3D position estimation, we address ground plane correction in real-time. The efficacy of the proposed method is demonstrated through experimental evaluations that compare our results to ground truth data available from other sensors including Can-Bus and LiDAR.

Related papers

• Multi-Object Tracking with Camera-LiDAR Fusion for Autonomous Driving [0.764971671709743]
  The proposed MOT algorithm comprises a three-step association process, an Extended Kalman filter for estimating the motion of each detected dynamic obstacle, and a track management phase. Unlike most state-of-the-art multi-modal MOT approaches, the proposed algorithm does not rely on maps or knowledge of the ego global pose. The algorithm is validated both in simulation and with real-world data, with satisfactory results.
  arXiv  Detail & Related papers  (2024-03-06T23:49:16Z)
• HUM3DIL: Semi-supervised Multi-modal 3D Human Pose Estimation for Autonomous Driving [95.42203932627102]
  3D human pose estimation is an emerging technology, which can enable the autonomous vehicle to perceive and understand the subtle and complex behaviors of pedestrians. Our method efficiently makes use of these complementary signals, in a semi-supervised fashion and outperforms existing methods with a large margin. Specifically, we embed LiDAR points into pixel-aligned multi-modal features, which we pass through a sequence of Transformer refinement stages.
  arXiv  Detail & Related papers  (2022-12-15T11:15:14Z)
• Ego Vehicle Speed Estimation using 3D Convolution with Masked Attention [0.0]
  We propose a novel 3D-CNN with masked-attention architecture to estimate ego vehicle speed. We conduct experiments on two publicly available datasets, nuImages and KITTI.
  arXiv  Detail & Related papers  (2022-12-11T07:22:25Z)
• Exploring Contextual Representation and Multi-Modality for End-to-End Autonomous Driving [58.879758550901364]
  Recent perception systems enhance spatial understanding with sensor fusion but often lack full environmental context. We introduce a framework that integrates three cameras to emulate the human field of view, coupled with top-down bird-eye-view semantic data to enhance contextual representation. Our method achieves displacement error by 0.67m in open-loop settings, surpassing current methods by 6.9% on the nuScenes dataset.
  arXiv  Detail & Related papers  (2022-10-13T05:56:20Z)
• Cyber Mobility Mirror for Enabling Cooperative Driving Automation: A Co-Simulation Platform [16.542137414609606]
  Co-simulation platform can simulate both the real world with a high-fidelity sensor perception system and the cyber world with a real-time 3D reconstruction system. Mirror-world simulator is responsible for reconstructing 3D objects and their trajectories from the perceived information. Roadside LiDAR-based real-time vehicle detection and 3D reconstruction system is prototyped as a study case.
  arXiv  Detail & Related papers  (2022-01-24T05:27:20Z)
• Real Time Monocular Vehicle Velocity Estimation using Synthetic Data [78.85123603488664]
  We look at the problem of estimating the velocity of road vehicles from a camera mounted on a moving car. We propose a two-step approach where first an off-the-shelf tracker is used to extract vehicle bounding boxes and then a small neural network is used to regress the vehicle velocity.
  arXiv  Detail & Related papers  (2021-09-16T13:10:27Z)
• CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking [9.62721286522053]
  We propose an end-to-end network for joint object detection and tracking based on radar and camera sensor fusion. Our proposed method uses a center-based radar-camera fusion algorithm for object detection and utilizes a greedy algorithm for object association. We evaluate our method on the challenging nuScenes dataset, where it achieves 20.0 AMOTA and outperforms all vision-based 3D tracking methods in the benchmark.
  arXiv  Detail & Related papers  (2021-07-11T23:56:53Z)
• Multi-Modal Fusion Transformer for End-to-End Autonomous Driving [59.60483620730437]
  We propose TransFuser, a novel Multi-Modal Fusion Transformer, to integrate image and LiDAR representations using attention. Our approach achieves state-of-the-art driving performance while reducing collisions by 76% compared to geometry-based fusion.
  arXiv  Detail & Related papers  (2021-04-19T11:48:13Z)
• Monocular 3D Vehicle Detection Using Uncalibrated Traffic Cameras through Homography [12.062095895630563]
  This paper proposes a method to extract the position and pose of vehicles in the 3D world from a single traffic camera. We observe that the homography between the road plane and the image plane is essential to 3D vehicle detection. We propose a new regression target called textittailedr-box and a textitdual-view network architecture which boosts the detection accuracy on warped BEV images.
  arXiv  Detail & Related papers  (2021-03-29T02:57:37Z)
• Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data Augmentation [77.60050239225086]
  We propose an effective training data generation process by fitting a 3D car model with dynamic parts to vehicles in real images. Our approach is fully automatic without any human interaction. We present a multi-task network for VUS parsing and a multi-stream network for VHI parsing.
  arXiv  Detail & Related papers  (2020-12-15T03:03:38Z)
• Towards Autonomous Driving: a Multi-Modal 360$^{\circ}$ Perception Proposal [87.11988786121447]
  This paper presents a framework for 3D object detection and tracking for autonomous vehicles. The solution, based on a novel sensor fusion configuration, provides accurate and reliable road environment detection. A variety of tests of the system, deployed in an autonomous vehicle, have successfully assessed the suitability of the proposed perception stack.
  arXiv  Detail & Related papers  (2020-08-21T20:36:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.