TOPO-Bench: An Open-Source Topological Mapping Evaluation Framework with Quantifiable Perceptual Aliasing

• URL: http://arxiv.org/abs/2510.04100v1
• Date: Sun, 05 Oct 2025 08:58:08 GMT
• Title: TOPO-Bench: An Open-Source Topological Mapping Evaluation Framework with Quantifiable Perceptual Aliasing
• Authors: Jiaming Wang, Diwen Liu, Jizhuo Chen, Harold Soh,
• Abstract summary: We formalize topological consistency as the fundamental property of topological maps and show that localization accuracy provides an efficient surrogate metric.<n>We propose the first quantitative measure of dataset ambiguity to enable fair comparisons across environments.<n>All datasets, baselines, and evaluation tools are fully open-sourced to foster consistent and reproducible research in topological mapping.
• Score: 10.736029638634504
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Topological mapping offers a compact and robust representation for navigation, but progress in the field is hindered by the lack of standardized evaluation metrics, datasets, and protocols. Existing systems are assessed using different environments and criteria, preventing fair and reproducible comparisons. Moreover, a key challenge - perceptual aliasing - remains under-quantified, despite its strong influence on system performance. We address these gaps by (1) formalizing topological consistency as the fundamental property of topological maps and showing that localization accuracy provides an efficient and interpretable surrogate metric, and (2) proposing the first quantitative measure of dataset ambiguity to enable fair comparisons across environments. To support this protocol, we curate a diverse benchmark dataset with calibrated ambiguity levels, implement and release deep-learned baseline systems, and evaluate them alongside classical methods. Our experiments and analysis yield new insights into the limitations of current approaches under perceptual aliasing. All datasets, baselines, and evaluation tools are fully open-sourced to foster consistent and reproducible research in topological mapping.

Related papers

• Place Recognition Meet Multiple Modalitie: A Comprehensive Review, Current Challenges and Future Directions [2.4775350526606355]
  We review recent advancements in place recognition, emphasizing three methodological paradigms.<n>CNN-based approaches, Transformer-based frameworks, and cross-modal strategies are discussed.<n>We identify current research challenges and outline prospective directions, including domain adaptation, real-time performance, and lifelong learning, to inspire future advancements in this domain.
  arXiv  Detail & Related papers  (2025-05-20T08:16:37Z)
• Benchmarking Synthetic Tabular Data: A Multi-Dimensional Evaluation Framework [0.4874819476581695]
  evaluating the quality of synthetic data remains a key challenge for ensuring privacy and utility in data-driven research.<n>We present a framework that quantifies how well synthetic data replicates original distributional properties while ensuring privacy.
  arXiv  Detail & Related papers  (2025-04-02T17:10:30Z)
• Map Feature Perception Metric for Map Generation Quality Assessment and Loss Optimization [2.311323886036968]
  This study introduces a novel Map Feature Metric designed to evaluate global characteristics and spatial congruence between synthesized and target maps.<n>Our approach extracts elemental-level deep features that comprehensively encode cartographic structural integrity and topological relationships.
  arXiv  Detail & Related papers  (2025-03-30T09:07:09Z)
• Out-of-Distribution Detection on Graphs: A Survey [58.47395497985277]
  Graph out-of-distribution (GOOD) detection focuses on identifying graph data that deviates from the distribution seen during training.<n>We categorize existing methods into four types: enhancement-based, reconstruction-based, information propagation-based, and classification-based approaches.<n>We discuss practical applications and theoretical foundations, highlighting the unique challenges posed by graph data.
  arXiv  Detail & Related papers  (2025-02-12T04:07:12Z)
• Pitfalls of topology-aware image segmentation [81.19923502845441]
  We identify critical pitfalls in model evaluation that include inadequate connectivity choices, overlooked topological artifacts, and inappropriate use of evaluation metrics.<n>We propose a set of actionable recommendations to establish fair and robust evaluation standards for topology-aware medical image segmentation methods.
  arXiv  Detail & Related papers  (2024-12-19T08:11:42Z)
• Topograph: An efficient Graph-Based Framework for Strictly Topology Preserving Image Segmentation [78.54656076915565]
  Topological correctness plays a critical role in many image segmentation tasks.<n>Most networks are trained using pixel-wise loss functions, such as Dice, neglecting topological accuracy.<n>We propose a novel, graph-based framework for topologically accurate image segmentation.
  arXiv  Detail & Related papers  (2024-11-05T16:20:14Z)
• On the Evaluation Consistency of Attribution-based Explanations [42.1421504321572]
  We introduce Meta-Rank, an open platform for benchmarking attribution methods in the image domain. Our benchmark reveals three insights in attribution evaluation endeavors: 1) evaluating attribution methods under disparate settings can yield divergent performance rankings; 2) although inconsistent across numerous cases, the performance rankings exhibit remarkable consistency across distinct checkpoints along the same training trajectory; and 3) prior attempts at consistent evaluation fare no better than baselines when extended to more heterogeneous models and datasets.
  arXiv  Detail & Related papers  (2024-07-28T11:49:06Z)
• View Consistent Purification for Accurate Cross-View Localization [59.48131378244399]
  This paper proposes a fine-grained self-localization method for outdoor robotics. The proposed method addresses limitations in existing cross-view localization methods. It is the first sparse visual-only method that enhances perception in dynamic environments.
  arXiv  Detail & Related papers  (2023-08-16T02:51:52Z)
• Adaptive Local-Component-aware Graph Convolutional Network for One-shot Skeleton-based Action Recognition [54.23513799338309]
  We present an Adaptive Local-Component-aware Graph Convolutional Network for skeleton-based action recognition. Our method provides a stronger representation than the global embedding and helps our model reach state-of-the-art.
  arXiv  Detail & Related papers  (2022-09-21T02:33:07Z)
• GenDICE: Generalized Offline Estimation of Stationary Values [108.17309783125398]
  We show that effective estimation can still be achieved in important applications. Our approach is based on estimating a ratio that corrects for the discrepancy between the stationary and empirical distributions. The resulting algorithm, GenDICE, is straightforward and effective.
  arXiv  Detail & Related papers  (2020-02-21T00:27:52Z)
• On clustering uncertain and structured data with Wasserstein barycenters and a geodesic criterion for the number of clusters [0.0]
  This work considers the notion of Wasserstein barycenters, accompanied by appropriate clustering indices based on the intrinsic geometry of the Wasserstein space where the clustering task is performed. Such type of clustering approaches are highly appreciated in many fields where the observational/experimental error is significant. Under this perspective, each observation is identified by an appropriate probability measure and the proposed clustering schemes rely on discrimination criteria.
  arXiv  Detail & Related papers  (2019-12-26T08:46:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.