CSAOT: Cooperative Multi-Agent System for Active Object Tracking

• URL: http://arxiv.org/abs/2501.13994v1
• Date: Thu, 23 Jan 2025 10:44:35 GMT
• Title: CSAOT: Cooperative Multi-Agent System for Active Object Tracking
• Authors: Hy Nguyen, Bao Pham, Hung Du, Srikanth Thudumu, Rajesh Vasa, Kon Mouzakis,
• Abstract summary: Active Object Tracking (AOT) requires a controller agent to actively adjust its viewpoint to maintain visual contact with a moving target in complex environments. Existing AOT solutions are predominantly single-agent-based, which struggle in dynamic and complex scenarios. We introduce the Collaborative System for Active Object Tracking (CSAOT) to enable multiple agents to operate on a single device.
• Score: 1.384468678066823
• License:
• Abstract: Object Tracking is essential for many computer vision applications, such as autonomous navigation, surveillance, and robotics. Unlike Passive Object Tracking (POT), which relies on static camera viewpoints to detect and track objects across consecutive frames, Active Object Tracking (AOT) requires a controller agent to actively adjust its viewpoint to maintain visual contact with a moving target in complex environments. Existing AOT solutions are predominantly single-agent-based, which struggle in dynamic and complex scenarios due to limited information gathering and processing capabilities, often resulting in suboptimal decision-making. Alleviating these limitations necessitates the development of a multi-agent system where different agents perform distinct roles and collaborate to enhance learning and robustness in dynamic and complex environments. Although some multi-agent approaches exist for AOT, they typically rely on external auxiliary agents, which require additional devices, making them costly. In contrast, we introduce the Collaborative System for Active Object Tracking (CSAOT), a method that leverages multi-agent deep reinforcement learning (MADRL) and a Mixture of Experts (MoE) framework to enable multiple agents to operate on a single device, thereby improving tracking performance and reducing costs. Our approach enhances robustness against occlusions and rapid motion while optimizing camera movements to extend tracking duration. We validated the effectiveness of CSAOT on various interactive maps with dynamic and stationary obstacles.

Related papers

• A Cross-Scene Benchmark for Open-World Drone Active Tracking [54.235808061746525]
  Drone Visual Active Tracking aims to autonomously follow a target object by controlling the motion system based on visual observations. We propose a unified cross-scene cross-domain benchmark for open-world drone active tracking called DAT. We also propose a reinforcement learning-based drone tracking method called R-VAT.
  arXiv  Detail & Related papers  (2024-12-01T09:37:46Z)
• Very Large-Scale Multi-Agent Simulation in AgentScope [112.98986800070581]
  We develop new features and components for AgentScope, a user-friendly multi-agent platform. We propose an actor-based distributed mechanism towards great scalability and high efficiency. We also provide a web-based interface for conveniently monitoring and managing a large number of agents.
  arXiv  Detail & Related papers  (2024-07-25T05:50:46Z)
• QuadrupedGPT: Towards a Versatile Quadruped Agent in Open-ended Worlds [51.05639500325598]
  We introduce QuadrupedGPT, designed to follow diverse commands with agility comparable to that of a pet. Our agent shows proficiency in handling diverse tasks and intricate instructions, representing a significant step toward the development of versatile quadruped agents.
  arXiv  Detail & Related papers  (2024-06-24T12:14:24Z)
• Track Anything Rapter(TAR) [0.0]
  Track Anything Rapter (TAR) is designed to detect, segment, and track objects of interest based on user-provided multimodal queries. TAR utilizes cutting-edge pre-trained models like DINO, CLIP, and SAM to estimate the relative pose of the queried object. We showcase how the integration of these foundational models with a custom high-level control algorithm results in a highly stable and precise tracking system.
  arXiv  Detail & Related papers  (2024-05-19T19:51:41Z)
• Tracking Transforming Objects: A Benchmark [2.53045657890708]
  This study collects a novel dedicated dataset for Tracking Transforming Objects, called DTTO, which contains 100 sequences, amounting to approximately 9.3K frames. We provide carefully hand-annotated bounding boxes for each frame within these sequences, making DTTO the pioneering benchmark dedicated to tracking transforming objects. We thoroughly evaluate 20 state-of-the-art trackers on the benchmark, aiming to comprehend the performance of existing methods and provide a comparison for future research on DTTO.
  arXiv  Detail & Related papers  (2024-04-28T11:24:32Z)
• AgentScope: A Flexible yet Robust Multi-Agent Platform [66.64116117163755]
  AgentScope is a developer-centric multi-agent platform with message exchange as its core communication mechanism. The abundant syntactic tools, built-in agents and service functions, user-friendly interfaces for application demonstration and utility monitor, zero-code programming workstation, and automatic prompt tuning mechanism significantly lower the barriers to both development and deployment.
  arXiv  Detail & Related papers  (2024-02-21T04:11:28Z)
• MotionTrack: Learning Robust Short-term and Long-term Motions for Multi-Object Tracking [56.92165669843006]
  We propose MotionTrack, which learns robust short-term and long-term motions in a unified framework to associate trajectories from a short to long range. For dense crowds, we design a novel Interaction Module to learn interaction-aware motions from short-term trajectories, which can estimate the complex movement of each target. For extreme occlusions, we build a novel Refind Module to learn reliable long-term motions from the target's history trajectory, which can link the interrupted trajectory with its corresponding detection.
  arXiv  Detail & Related papers  (2023-03-18T12:38:33Z)
• Scalable and Real-time Multi-Camera Vehicle Detection, Re-Identification, and Tracking [58.95210121654722]
  We propose a real-time city-scale multi-camera vehicle tracking system that handles real-world, low-resolution CCTV instead of idealized and curated video streams. Our method is ranked among the top five performers on the public leaderboard.
  arXiv  Detail & Related papers  (2022-04-15T12:47:01Z)
• Multi-target tracking for video surveillance using deep affinity network: a brief review [0.0]
  Multi-target tracking (MTT) for video surveillance is one of the important and challenging tasks. Deep learning models are known to function like the human brain.
  arXiv  Detail & Related papers  (2021-10-29T10:44:26Z)
• Multi-Agent Embodied Visual Semantic Navigation with Scene Prior Knowledge [42.37872230561632]
  In visual semantic navigation, the robot navigates to a target object with egocentric visual observations and the class label of the target is given. Most of the existing models are only effective for single-agent navigation, and a single agent has low efficiency and poor fault tolerance when completing more complicated tasks. We propose the multi-agent visual semantic navigation, in which multiple agents collaborate with others to find multiple target objects.
  arXiv  Detail & Related papers  (2021-09-20T13:31:03Z)
• Distributed Reinforcement Learning of Targeted Grasping with Active Vision for Mobile Manipulators [4.317864702902075]
  We present the first RL-based system for a mobile manipulator that can (a) achieve targeted grasping generalizing to unseen target objects, (b) learn complex grasping strategies for cluttered scenes with occluded objects, and (c) perform active vision through its movable wrist camera to better locate objects. We train and evaluate our system in a simulated environment, identify key components for improving performance, analyze its behaviors, and transfer to a real-world setup.
  arXiv  Detail & Related papers  (2020-07-16T02:47:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.