ObjectReact: Learning Object-Relative Control for Visual Navigation

• URL: http://arxiv.org/abs/2509.09594v1
• Date: Thu, 11 Sep 2025 16:34:17 GMT
• Title: ObjectReact: Learning Object-Relative Control for Visual Navigation
• Abstract summary: We present a new paradigm of learning "object-relative" control that exhibits several desirable characteristics.<n>We propose a topometric map representation in the form of a "relative" 3D scene graph.<n>We demonstrate the advantages of learning object-relative control over its image-relative counterpart across sensor height variations.
• Score: 10.645109937081681
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual navigation using only a single camera and a topological map has recently become an appealing alternative to methods that require additional sensors and 3D maps. This is typically achieved through an "image-relative" approach to estimating control from a given pair of current observation and subgoal image. However, image-level representations of the world have limitations because images are strictly tied to the agent's pose and embodiment. In contrast, objects, being a property of the map, offer an embodiment- and trajectory-invariant world representation. In this work, we present a new paradigm of learning "object-relative" control that exhibits several desirable characteristics: a) new routes can be traversed without strictly requiring to imitate prior experience, b) the control prediction problem can be decoupled from solving the image matching problem, and c) high invariance can be achieved in cross-embodiment deployment for variations across both training-testing and mapping-execution settings. We propose a topometric map representation in the form of a "relative" 3D scene graph, which is used to obtain more informative object-level global path planning costs. We train a local controller, dubbed "ObjectReact", conditioned directly on a high-level "WayObject Costmap" representation that eliminates the need for an explicit RGB input. We demonstrate the advantages of learning object-relative control over its image-relative counterpart across sensor height variations and multiple navigation tasks that challenge the underlying spatial understanding capability, e.g., navigating a map trajectory in the reverse direction. We further show that our sim-only policy is able to generalize well to real-world indoor environments. Code and supplementary material are accessible via project page: https://object-react.github.io/

Related papers

• Flying to Image-Specified Objects: 3D Quadrotor Navigation via Cross-Graph Memory and Viewpoint Planning [6.546667407424349]
  Instance-Specific Image-Goal Navigation (InstanceImageNav) requires a robot to navigate toward the exact object instance depicted in a query image.<n>We propose a hierarchical navigation framework for quadrotor InstanceImageNav that separates high-level decision making from low-level motion execution.
  arXiv  Detail & Related papers  (2026-06-29T07:54:24Z)
• MASt3R-Nav: WayPixel Navigation in Relative 3D Maps [21.489070961711548]
  We propose a novel map representation in the form of pixel-relative connectivity.<n>Inspired by recent progress in 3D grounded image matching, we construct a map from an image sequence.<n>We show that this dense pixel-level costmap based on relative geometry is a more accurate conditioning variable for control prediction than its image- and object-level counterparts.
  arXiv  Detail & Related papers  (2026-05-22T18:18:07Z)
• Context-Nav: Context-Driven Exploration and Viewpoint-Aware 3D Spatial Reasoning for Instance Navigation [4.039245878626346]
  Text-goal instance navigation (TGIN) asks an agent to resolve a single, free-form description into actions that reach the correct object instance.<n>We present textitContext-Nav, which elevates long, contextual captions from a local cue to a global exploration prior.
  arXiv  Detail & Related papers  (2026-03-10T11:08:35Z)
• FeudalNav: A Simple Framework for Visual Navigation [7.136542835931238]
  We develop a hierarchical framework that decomposes the navigation decision-making process into multiple levels.<n>Our method learns to select subgoals through a simple, transferable waypoint selection network.<n>We show competitive results with a suite of SOTA methods in Habitat AI environments without using any odometry in training or inference.
  arXiv  Detail & Related papers  (2026-01-15T22:10:29Z)
• Compass Control: Multi Object Orientation Control for Text-to-Image Generation [24.4172525865806]
  Existing approaches for controlling text-to-image diffusion models, while powerful, do not allow for explicit 3D object-centric control.<n>We address the problem of multi-object orientation control in text-to-image diffusion models.<n>This enables the generation of diverse multi-object scenes with precise orientation control for each object.
  arXiv  Detail & Related papers  (2025-04-09T10:15:15Z)
• GaussNav: Gaussian Splatting for Visual Navigation [92.13664084464514]
  Instance ImageGoal Navigation (IIN) requires an agent to locate a specific object depicted in a goal image within an unexplored environment.<n>We propose a new framework for IIN, Gaussian Splatting for Visual Navigation (GaussNav), which constructs a novel map representation based on 3D Gaussian Splatting (3DGS)<n>Our GaussNav framework demonstrates a significant performance improvement, with Success weighted by Path Length (SPL) increasing from 0.347 to 0.578 on the challenging Habitat-Matterport 3D (HM3D) dataset.
  arXiv  Detail & Related papers  (2024-03-18T09:56:48Z)
• Instance-aware Exploration-Verification-Exploitation for Instance ImageGoal Navigation [88.84058353659107]
  Instance ImageGoal Navigation (IIN) aims to navigate to a specified object depicted by a goal image in an unexplored environment. We propose a new modular navigation framework named Instance-aware Exploration-Verification-Exploitation (IEVE) for instance-level image goal navigation. Our method surpasses previous state-of-the-art work, with a classical segmentation model (0.684 vs. 0.561 success) or a robust model (0.702 vs. 0.561 success)
  arXiv  Detail & Related papers  (2024-02-25T07:59:10Z)
• Towards Generalizable Multi-Camera 3D Object Detection via Perspective Debiasing [28.874014617259935]
  Multi-Camera 3D Object Detection (MC3D-Det) has gained prominence with the advent of bird's-eye view (BEV) approaches. We propose a novel method that aligns 3D detection with 2D camera plane results, ensuring consistent and accurate detections.
  arXiv  Detail & Related papers  (2023-10-17T15:31:28Z)
• Object Goal Navigation with Recursive Implicit Maps [92.6347010295396]
  We propose an implicit spatial map for object goal navigation. Our method significantly outperforms the state of the art on the challenging MP3D dataset. We deploy our model on a real robot and achieve encouraging object goal navigation results in real scenes.
  arXiv  Detail & Related papers  (2023-08-10T14:21:33Z)
• DECA: Deep viewpoint-Equivariant human pose estimation using Capsule Autoencoders [3.2826250607043796]
  We show that current 3D Human Pose Estimation methods tend to fail when dealing with viewpoints unseen at training time. We propose a novel capsule autoencoder network with fast Variational Bayes capsule routing, named DECA. In the experimental validation, we outperform other methods on depth images from both seen and unseen viewpoints, both top-view, and front-view.
  arXiv  Detail & Related papers  (2021-08-19T08:46:15Z)
• Progressive Coordinate Transforms for Monocular 3D Object Detection [52.00071336733109]
  We propose a novel and lightweight approach, dubbed em Progressive Coordinate Transforms (PCT) to facilitate learning coordinate representations. In this paper, we propose a novel and lightweight approach, dubbed em Progressive Coordinate Transforms (PCT) to facilitate learning coordinate representations.
  arXiv  Detail & Related papers  (2021-08-12T15:22:33Z)
• Sparse Pose Trajectory Completion [87.31270669154452]
  We propose a method to learn, even using a dataset where objects appear only in sparsely sampled views. This is achieved with a cross-modal pose trajectory transfer mechanism. Our method is evaluated on the Pix3D and ShapeNet datasets.
  arXiv  Detail & Related papers  (2021-05-01T00:07:21Z)
• SOON: Scenario Oriented Object Navigation with Graph-based Exploration [102.74649829684617]
  The ability to navigate like a human towards a language-guided target from anywhere in a 3D embodied environment is one of the 'holy grail' goals of intelligent robots. Most visual navigation benchmarks focus on navigating toward a target from a fixed starting point, guided by an elaborate set of instructions that depicts step-by-step. This approach deviates from real-world problems in which human-only describes what the object and its surrounding look like and asks the robot to start navigation from anywhere.
  arXiv  Detail & Related papers  (2021-03-31T15:01:04Z)
• Memory-Augmented Reinforcement Learning for Image-Goal Navigation [67.3963444878746]
  We present a novel method that leverages a cross-episode memory to learn to navigate. In order to avoid overfitting, we propose to use data augmentation on the RGB input during training. We obtain this competitive performance from RGB input only, without access to additional sensors such as position or depth.
  arXiv  Detail & Related papers  (2021-01-13T16:30:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.