Beyond Chamfer Distance: Granular Order-aware Evaluation Metric For Online Mapping

• URL: http://arxiv.org/abs/2605.22578v1
• Date: Thu, 21 May 2026 14:51:52 GMT
• Title: Beyond Chamfer Distance: Granular Order-aware Evaluation Metric For Online Mapping
• Abstract summary: State-of-the-art (SOTA) methods commonly predict map elements as ordered sequences of points that form polylines and polygons.<n>This framework lacks sensitivity to point ordering and provides limited granularity in assessing geometric quality.<n>We introduce sequence optimal sub-pattern assignment (SOSPA), an order-aware metric that enables fine-grained evaluation of individual geometries.
• Score: 9.868903360882262
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Online map estimation is a crucial component of autonomous driving systems that reduces the reliance on costly high-definition maps. State-of-the-art (SOTA) methods commonly predict map elements as ordered sequences of points that form polylines and polygons. The evaluation of these methods relies predominantly on mean average precision (mAP) based on thresholded Chamfer distance (CD). This framework lacks sensitivity to point ordering and provides limited granularity in assessing geometric quality, making it difficult to distinguish which methods truly excel over others. In this work, we address these limitations on two fronts. For the single-instance similarity measure, we introduce sequence optimal sub-pattern assignment (SOSPA), an order-aware metric that enables fine-grained evaluation of individual geometries while satisfying all metric axioms. For the multi-instance evaluation framework, we propose polyline localisation and detection (PLD), a soft metric that jointly captures detection quality and geometric accuracy, replacing the hard thresholding of mAP with a principled soft assignment. Through evaluations on nuScenes, we demonstrate that PLD effectively ranks SOTA online mapping methods (MapTRv2, StreamMapNet, MapTracker) while providing a decomposed error analysis. This analysis identifies detection capability as the dominant bottleneck in current methods, revealing a performance trend that mAP fails to capture. Code for evaluation using our metrics will be released.

Related papers

• CT-VoxelMap: Efficient Continuous-Time LiDAR-Inertial Odometry with Probabilistic Adaptive Voxel Mapping [15.34037575218496]
  We propose a new continuous-time odometry method based on the cumulative form of B-splines.<n>We use forward propagation information from IMU measurements to estimate fitting errors online and introduce a hybrid feature-based voxel map management strategy.<n>The proposed method is evaluated on multiple challenging public datasets, demonstrating superior performance on most sequences.
  arXiv  Detail & Related papers  (2026-04-04T14:27:54Z)
• Failure Modes for Deep Learning-Based Online Mapping: How to Measure and Address Them [10.503307305047487]
  We propose a framework to identify and measure the underlying failure modes by disentangling two effects: Memorization of input features and overfitting to known map geometries.<n>We introduce Fréchet distance-based reconstruction statistics that capture per-element shape fidelity without threshold tuning.<n>We formulate an MST-based sparsification strategy that reduces redundancy and improves balancing and performance while shrinking training size.
  arXiv  Detail & Related papers  (2026-03-20T11:09:11Z)
• MS-ISSM: Objective Quality Assessment of Point Clouds Using Multi-scale Implicit Structural Similarity [65.85858856481131]
  unstructured and irregular nature of point clouds poses a significant challenge for objective quality assessment (PCQA)<n>We propose the Multi-scale Implicit Structural Similarity Measurement (MS-ISSM)
  arXiv  Detail & Related papers  (2026-01-03T14:58:52Z)
• Robust Nearest Neighbour Retrieval Using Targeted Manifold Manipulation [0.0]
  Nearest-neighbour retrieval is central to classification and explainable-AI pipelines.<n>We propose Targeted Manifold Manipulation-Nearest Neighbour (TMM-NN), which reconceptualises retrieval by assessing how readily each sample can be nudged into a designated region of the feature manifold.<n>TMM-NN implements this through a lightweight, query-specific trigger patch.
  arXiv  Detail & Related papers  (2025-11-09T07:37:05Z)
• Piecewise-Linear Manifolds for Deep Metric Learning [8.670873561640903]
  Unsupervised deep metric learning focuses on learning a semantic representation space using only unlabeled data. We propose to model the high-dimensional data manifold using a piecewise-linear approximation, with each low-dimensional linear piece approximating the data manifold in a small neighborhood of a point. We empirically show that this similarity estimate correlates better with the ground truth than the similarity estimates of current state-of-the-art techniques.
  arXiv  Detail & Related papers  (2024-03-22T06:22:20Z)
• Rethinking Counting and Localization in Crowds:A Purely Point-Based Framework [59.578339075658995]
  We propose a purely point-based framework for joint crowd counting and individual localization. We design an intuitive solution under this framework, which is called Point to Point Network (P2PNet)
  arXiv  Detail & Related papers  (2021-07-27T11:41:50Z)
• Improving Metric Dimensionality Reduction with Distributed Topology [68.8204255655161]
  DIPOLE is a dimensionality-reduction post-processing step that corrects an initial embedding by minimizing a loss functional with both a local, metric term and a global, topological term. We observe that DIPOLE outperforms popular methods like UMAP, t-SNE, and Isomap on a number of popular datasets.
  arXiv  Detail & Related papers  (2021-06-14T17:19:44Z)
• Making Affine Correspondences Work in Camera Geometry Computation [62.7633180470428]
  Local features provide region-to-region rather than point-to-point correspondences. We propose guidelines for effective use of region-to-region matches in the course of a full model estimation pipeline. Experiments show that affine solvers can achieve accuracy comparable to point-based solvers at faster run-times.
  arXiv  Detail & Related papers  (2020-07-20T12:07:48Z)
• Rethinking Localization Map: Towards Accurate Object Perception with Self-Enhancement Maps [78.2581910688094]
  This work introduces a novel self-enhancement method to harvest accurate object localization maps and object boundaries with only category labels as supervision. In particular, the proposed Self-Enhancement Maps achieve the state-of-the-art localization accuracy of 54.88% on ILSVRC.
  arXiv  Detail & Related papers  (2020-06-09T12:35:55Z)
• Learning Flat Latent Manifolds with VAEs [16.725880610265378]
  We propose an extension to the framework of variational auto-encoders, where the Euclidean metric is a proxy for the similarity between data points. We replace the compact prior typically used in variational auto-encoders with a recently presented, more expressive hierarchical one. We evaluate our method on a range of data-sets, including a video-tracking benchmark.
  arXiv  Detail & Related papers  (2020-02-12T09:54:52Z)
• CONSAC: Robust Multi-Model Fitting by Conditional Sample Consensus [62.86856923633923]
  We present a robust estimator for fitting multiple parametric models of the same form to noisy measurements. In contrast to previous works, which resorted to hand-crafted search strategies for multiple model detection, we learn the search strategy from data. For self-supervised learning of the search, we evaluate the proposed algorithm on multi-homography estimation and demonstrate an accuracy that is superior to state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-01-08T17:37:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.