Coordinate-Aligned Multi-Camera Collaboration for Active Multi-Object Tracking

• URL: http://arxiv.org/abs/2202.10881v1
• Date: Tue, 22 Feb 2022 13:28:40 GMT
• Title: Coordinate-Aligned Multi-Camera Collaboration for Active Multi-Object Tracking
• Authors: Zeyu Fang, Jian Zhao, Mingyu Yang, Wengang Zhou, Zhenbo Lu, Houqiang Li
• Abstract summary: We propose a coordinate-aligned multi-camera collaboration system for AMOT. In our approach, we regard each camera as an agent and address AMOT with a multi-agent reinforcement learning solution. Our system achieves a coverage of 71.88%, outperforming the baseline method by 8.9%.
• Score: 114.16306938870055
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Active Multi-Object Tracking (AMOT) is a task where cameras are controlled by a centralized system to adjust their poses automatically and collaboratively so as to maximize the coverage of targets in their shared visual field. In AMOT, each camera only receives partial information from its observation, which may mislead cameras to take locally optimal action. Besides, the global goal, i.e., maximum coverage of objects, is hard to be directly optimized. To address the above issues, we propose a coordinate-aligned multi-camera collaboration system for AMOT. In our approach, we regard each camera as an agent and address AMOT with a multi-agent reinforcement learning solution. To represent the observation of each agent, we first identify the targets in the camera view with an image detector, and then align the coordinates of the targets in 3D environment. We define the reward of each agent based on both global coverage as well as four individual reward terms. The action policy of the agents is derived with a value-based Q-network. To the best of our knowledge, we are the first to study the AMOT task. To train and evaluate the efficacy of our system, we build a virtual yet credible 3D environment, named "Soccer Court", to mimic the real-world AMOT scenario. The experimental results show that our system achieves a coverage of 71.88%, outperforming the baseline method by 8.9%.

Related papers

• PoIFusion: Multi-Modal 3D Object Detection via Fusion at Points of Interest [65.48057241587398]
  PoIFusion is a framework to fuse information of RGB images and LiDAR point clouds at the points of interest (PoIs) Our approach maintains the view of each modality and obtains multi-modal features by computation-friendly projection and computation. We conducted extensive experiments on nuScenes and Argoverse2 datasets to evaluate our approach.
  arXiv  Detail & Related papers  (2024-03-14T09:28:12Z)
• Collaboration Helps Camera Overtake LiDAR in 3D Detection [49.58433319402405]
  Camera-only 3D detection provides a simple solution for localizing objects in 3D space compared to LiDAR-based detection systems. Our proposed collaborative camera-only 3D detection (CoCa3D) enables agents to share complementary information with each other through communication. Results show that CoCa3D improves previous SOTA performances by 44.21% on DAIR-V2X, 30.60% on OPV2V+, 12.59% on CoPerception-UAVs+ for AP@70.
  arXiv  Detail & Related papers  (2023-03-23T03:50:41Z)
• Multi-Camera Multi-Object Tracking on the Move via Single-Stage Global Association Approach [23.960847268459293]
  This work introduces novel Single-Stage Global Association Tracking approaches to associate one or more detection from multi-cameras with tracked objects. Our models also improve the detection accuracy of the standard vision-based 3D object detectors in the nuScenes detection challenge.
  arXiv  Detail & Related papers  (2022-11-17T17:03:24Z)
• A Simple Baseline for Multi-Camera 3D Object Detection [94.63944826540491]
  3D object detection with surrounding cameras has been a promising direction for autonomous driving. We present SimMOD, a Simple baseline for Multi-camera Object Detection. We conduct extensive experiments on the 3D object detection benchmark of nuScenes to demonstrate the effectiveness of SimMOD.
  arXiv  Detail & Related papers  (2022-08-22T03:38:01Z)
• Multi-Target Active Object Tracking with Monte Carlo Tree Search and Target Motion Modeling [126.26121580486289]
  In this work, we are dedicated to multi-target active object tracking (AOT), where there are multiple targets as well as multiple cameras in the environment. The goal is maximize the overall target coverage of all cameras. We establish a multi-target 2D environment to simulate the sports games, and experimental results demonstrate that our method can effectively improve the target coverage.
  arXiv  Detail & Related papers  (2022-05-07T05:08:15Z)
• Object Manipulation via Visual Target Localization [64.05939029132394]
  Training agents to manipulate objects, poses many challenges. We propose an approach that explores the environment in search for target objects, computes their 3D coordinates once they are located, and then continues to estimate their 3D locations even when the objects are not visible. Our evaluations show a massive 3x improvement in success rate over a model that has access to the same sensory suite.
  arXiv  Detail & Related papers  (2022-03-15T17:59:01Z)
• Pose-Assisted Multi-Camera Collaboration for Active Object Tracking [42.57706021569103]
  Active Object Tracking (AOT) is crucial to many visionbased applications, e.g., mobile robot, intelligent surveillance. In this paper, we extend the single-camera AOT to a multi-camera setting, where cameras tracking a target in a collaborative fashion. We propose a novel Pose-Assisted Multi-Camera Collaboration System, which enables a camera to cooperate with the others by sharing camera poses for active object tracking.
  arXiv  Detail & Related papers  (2020-01-15T07:49:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.