Cross-Modal Benchmarking for Robotic Perception in Natural Environments

• URL: http://arxiv.org/abs/2606.11563v1
• Date: Wed, 10 Jun 2026 01:43:24 GMT
• Title: Cross-Modal Benchmarking for Robotic Perception in Natural Environments
• Abstract summary: Current models, particularly vision foundation models, are largely trained on structured, urban environments.<n>We showcase the limitations of current models using our recently released WildCross benchmark.
• Score: 6.143141555930762
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Natural environments present a complex challenge to robotics perception systems. Current models, particularly vision foundation models, are largely trained on structured, urban environments leading to weaknesses in their perception for field robotics tasks. We showcase the limitations of current models using our recently released WildCross benchmark, a new cross-modal benchmark for place recognition and metric depth estimation in large-scale natural environments. WildCross comprises over 476K sequential RGB frames with semi-dense depth and surface normal annotations, each aligned with accurate 6DoF pose and synchronized dense lidar submaps. In this work, we provide an expanded analysis of the benchmark results from the recent WildCross benchmark, with particular emphasis on expanded metric depth estimation experiments. Access to the code repository and dataset for this work can be found at https://csiro-robotics.github.io/WildCross.

Related papers

• SpatialBench: Is Your Spatial Foundation Model an All-Round Player? [92.031716744172]
  SpatialBench is a cross-paradigm, domain-diverse benchmark for spatial foundation models with deterministic sampling.<n>It comprehensively evaluates 41 models across 6 paradigms on 5 task suites under 4 different input density settings.<n>Our evaluations in challenging embodied and egocentric tasks demonstrate that strict domain alignment and high data quality are far more critical to performance than simple dataset scaling.
  arXiv  Detail & Related papers  (2026-05-26T17:59:20Z)
• WildCross: A Cross-Modal Large Scale Benchmark for Place Recognition and Metric Depth Estimation in Natural Environments [11.037873142796682]
  WildCross is a cross-modal benchmark for place recognition and metric depth estimation in large-scale natural environments.<n>We conduct comprehensive experiments on visual, lidar, and cross-modal place recognition, as well as metric depth estimation.
  arXiv  Detail & Related papers  (2026-03-02T05:39:12Z)
• GrandTour: A Legged Robotics Dataset in the Wild for Multi-Modal Perception and State Estimation [8.483668967252422]
  The GrandTour dataset is a multi-modal legged-robotics dataset collected across challenging outdoor and indoor environments.<n>The dataset provides time-synchronized sensor data from spinning LiDARs, multiple RGB cameras with complementary characteristics, proprioceptive sensors, and stereo depth cameras.<n>This dataset supports research in SLAM, high-precision state estimation, and multi-modal learning, enabling rigorous evaluation and development of new approaches to sensor fusion in legged robotic systems.
  arXiv  Detail & Related papers  (2026-02-20T11:57:52Z)
• Depth Completion in Unseen Field Robotics Environments Using Extremely Sparse Depth Measurements [17.714237831766173]
  We propose a depth completion model that trains on synthetic data and uses extremely sparse measurements from depth sensors to predict dense metric depth in unseen field robotics environments.<n>Our approach achieves an end-to-end latency of 53 ms per frame on a Nvidia Jetson AGX Orin, enabling real-time deployment on embedded platforms.
  arXiv  Detail & Related papers  (2026-02-03T07:24:05Z)
• DeFM: Learning Foundation Representations from Depth for Robotics [49.77188649197404]
  We present DeFM, a self-supervised foundation model trained entirely on depth images for robotic applications.<n>DeFM learns geometric and semantic representations that generalize to diverse environments, tasks, and sensors.<n>It achieves state-of-the-art performance and demonstrates strong generalization from simulation to real-world environments.
  arXiv  Detail & Related papers  (2026-01-26T19:45:31Z)
• ROVR-Open-Dataset: A Large-Scale Depth Dataset for Autonomous Driving [62.9051914830949]
  We present ROVR, a large-scale, diverse, and cost-efficient depth dataset designed to capture the complexity of real-world driving.<n>A lightweight acquisition pipeline ensures scalable collection, while sparse but statistically sufficient ground truth supports robust training.<n> Benchmarking with state-of-the-art monocular depth models reveals severe cross-dataset generalization failures.
  arXiv  Detail & Related papers  (2025-08-19T16:13:49Z)
• OpenEarthSensing: Large-Scale Fine-Grained Benchmark for Open-World Remote Sensing [57.050679160659705]
  We introduce textbfOpenEarthSensing (OES), a large-scale fine-grained benchmark for open-world remote sensing.<n>OES includes 189 scene and object categories, covering the vast majority of potential semantic shifts that may occur in the real world.
  arXiv  Detail & Related papers  (2025-02-28T02:49:52Z)
• MambaVT: Spatio-Temporal Contextual Modeling for robust RGB-T Tracking [51.28485682954006]
  We propose a pure Mamba-based framework (MambaVT) to fully exploit intrinsic-temporal contextual modeling for robust visible-thermal tracking. Specifically, we devise the long-range cross-frame integration component to globally adapt to target appearance variations. Experiments show the significant potential of vision Mamba for RGB-T tracking, with MambaVT achieving state-of-the-art performance on four mainstream benchmarks.
  arXiv  Detail & Related papers  (2024-08-15T02:29:00Z)
• Gait Recognition in the Wild: A Large-scale Benchmark and NAS-based Baseline [95.88825497452716]
  Gait benchmarks empower the research community to train and evaluate high-performance gait recognition systems. GREW is the first large-scale dataset for gait recognition in the wild. SPOSGait is the first NAS-based gait recognition model.
  arXiv  Detail & Related papers  (2022-05-05T14:57:39Z)
• SVIRO: Synthetic Vehicle Interior Rear Seat Occupancy Dataset and Benchmark [11.101588888002045]
  We release SVIRO, a synthetic dataset for sceneries in the passenger compartment of ten different vehicles. We analyze machine learning-based approaches for their generalization capacities and reliability when trained on a limited number of variations.
  arXiv  Detail & Related papers  (2020-01-10T14:44:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.