Temporally Consistent Unsupervised Segmentation for Mobile Robot Perception

• URL: http://arxiv.org/abs/2507.22194v1
• Date: Tue, 29 Jul 2025 19:41:37 GMT
• Title: Temporally Consistent Unsupervised Segmentation for Mobile Robot Perception
• Abstract summary: We introduce Frontier-Seg, a method for temporally consistent unsupervised segmentation of terrain from mobile robot video streams.<n>We evaluate Frontier-Seg on a diverse set of benchmark datasets-including RUGD and RELLIS-3D-demonstrating its ability to perform unsupervised segmentation across unstructured off-road environments.
• Score: 1.1749564892273827
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Rapid progress in terrain-aware autonomous ground navigation has been driven by advances in supervised semantic segmentation. However, these methods rely on costly data collection and labor-intensive ground truth labeling to train deep models. Furthermore, autonomous systems are increasingly deployed in unrehearsed, unstructured environments where no labeled data exists and semantic categories may be ambiguous or domain-specific. Recent zero-shot approaches to unsupervised segmentation have shown promise in such settings but typically operate on individual frames, lacking temporal consistency-a critical property for robust perception in unstructured environments. To address this gap we introduce Frontier-Seg, a method for temporally consistent unsupervised segmentation of terrain from mobile robot video streams. Frontier-Seg clusters superpixel-level features extracted from foundation model backbones-specifically DINOv2-and enforces temporal consistency across frames to identify persistent terrain boundaries or frontiers without human supervision. We evaluate Frontier-Seg on a diverse set of benchmark datasets-including RUGD and RELLIS-3D-demonstrating its ability to perform unsupervised segmentation across unstructured off-road environments.

Related papers

• Toward Robust LiDAR Semantic Segmentation for Real-World Deployment: Evaluation under Coarse Labels, Adverse Conditions, and Domain Shifts [39.58684038370709]
  LiDAR-based semantic segmentation is a core perception module for autonomous vehicles and mobile robots.<n>Existing evaluation protocols focus on clean, single-domain settings and fine-grained label robustness, leaving deployment readiness largely unassessed.<n>We propose a structured evaluation protocol that assesses the deployment readiness of LiDAR semantic segmentation models along three complementary dimensions.
  arXiv  Detail & Related papers  (2026-09-02T17:15:07Z)
• Prior-SG: Task and Prior Driven Region Segmentation for Scene Graphs in Arbitrarily-Structured Environments [20.027960785504586]
  We propose Prior-SG, a task- and prior-driven framework that casts scene graph generation as a probabilistic alignment problem.<n>We validate this approach across diverse simulated residential datasets and large, open-plan real-world environments.
  arXiv  Detail & Related papers  (2026-08-06T15:36:12Z)
• Modular Diffusion Models for Structured Visual Recognition [52.38687800325397]
  We present Modular Diffusion Models (MDMs), a simple and novel framework that learns a distribution over structured outputs for a given input image.<n>MDMs decompose the diffusion process into distinct, task-specific modules, each focused on capturing a different aspect of the structured information space.<n>This modular design allows each component to be learned independently, with seamless integration at inference without additional training.
  arXiv  Detail & Related papers  (2026-06-21T22:38:35Z)
• AnchorMoE: Interpretable Time Series Classification via Anchor-Routed MoE [51.994752158953084]
  AnchorMoE is an interpretable-by-construction classification framework.<n>It encodes multi-view representations of local patches and routes them to specialized experts.<n>Experiments on real-world and synthetic benchmarks demonstrate that AnchorMoE achieves highly competitive classification performance.
  arXiv  Detail & Related papers  (2026-06-02T13:30:54Z)
• Trinity: Unifying Class-Agnostic Terrain and Semantic Segmentation for Unstructured Outdoor Environments by Leveraging Synthetic Data [20.795406392781647]
  Existing vision-based traversability estimation methods rely on robot-specific annotations or semantic class mappings.<n>We propose a transformer-based architecture that jointly performs class-specific semantic segmentation and class-agnostic terrain segmentation.<n>We present the EXTerra dataset, providing real-world images annotated with both class-specific and class-agnostic terrain labels.
  arXiv  Detail & Related papers  (2026-05-26T20:04:19Z)
• HEAT: Heterogeneous End-to-End Autonomous Driving via Trajectory-Guided World Models [50.18051979020699]
  End-to-end autonomous driving has emerged as a compelling alternative to traditional modular pipelines.<n>We propose a trajectory-driven learning paradigm that organizes training around planning trajectories.<n>We show that a single unified model can be trained on heterogeneous datasets while maintaining strong performance within each domain.
  arXiv  Detail & Related papers  (2026-05-19T10:12:01Z)
• Instance-Guided Unsupervised Domain Adaptation for Robotic Semantic Segmentation [4.556831252263135]
  Unsupervised Domain Adaptation (UDA) adapts the network to the robot's target environment without external supervision.<n>We propose a method that starts from a 3D map to generate multi-view consistent pseudo-labels.<n>We refine these labels using the zero-shot instance segmentation capabilities of a foundation model, enforcing instance-level coherence.
  arXiv  Detail & Related papers  (2026-02-01T18:49:03Z)
• AURASeg: Attention Guided Upsampling with Residual Boundary-Assistive Refinement for Drivable-Area Segmentation [0.5729426778193399]
  We propose Attention-Guided Upsampling with Residual Boundary-Assistive Refinement (AURASeg) for semantic segmentation.<n>Our approach achieves an improvement in mean Intersection-over-Union (mIoU) of +1.26% and segmentation precision of +1.65% compared to state-of-the-art models.
  arXiv  Detail & Related papers  (2025-10-24T15:01:18Z)
• Embodied-RAG: General Non-parametric Embodied Memory for Retrieval and Generation [69.01029651113386]
  Embodied-RAG is a framework that enhances the model of an embodied agent with a non-parametric memory system.<n>At its core, Embodied-RAG's memory is structured as a semantic forest, storing language descriptions at varying levels of detail.<n>We demonstrate that Embodied-RAG effectively bridges RAG to the robotics domain, successfully handling over 250 explanation and navigation queries.
  arXiv  Detail & Related papers  (2024-09-26T21:44:11Z)
• OoDIS: Anomaly Instance Segmentation and Detection Benchmark [57.89836988990543]
  This work extends some commonly used anomaly segmentation benchmarks to include the instance segmentation and object detection tasks.<n>Our evaluation of anomaly segmentation and object detection methods shows that both of these challenges remain unsolved problems.
  arXiv  Detail & Related papers  (2024-06-17T17:59:56Z)
• Outlier-Robust Long-Term Robotic Mapping Leveraging Ground Segmentation [1.7948767405202701]
  I propose a robust long-term robotic mapping system that can work out of the box. I propose (i) fast and robust ground segmentation to reject the presence of outliers. I propose (ii)-robust registration with ground segmentation that encompasses the presence of gross outliers.
  arXiv  Detail & Related papers  (2024-05-18T04:56:15Z)
• Semi-Supervised Active Learning for Semantic Segmentation in Unknown Environments Using Informative Path Planning [27.460481202195012]
  Self-supervised and fully supervised active learning methods emerged to improve a robot's vision. We propose a planning method for semi-supervised active learning of semantic segmentation. We leverage an adaptive map-based planner guided towards the frontiers of unexplored space with high model uncertainty.
  arXiv  Detail & Related papers  (2023-12-07T16:16:47Z)
• A Spatiotemporal Correspondence Approach to Unsupervised LiDAR Segmentation with Traffic Applications [16.260518238832887]
  Key idea is to leverage the nature of a dynamic point cloud sequence and introduce drastically stronger scenarios. We alternate between optimizing semantic into groups and clustering using point-wisetemporal labels. Our method can learn discriminative features in an unsupervised learning fashion.
  arXiv  Detail & Related papers  (2023-08-23T21:32:46Z)
• A Threefold Review on Deep Semantic Segmentation: Efficiency-oriented, Temporal and Depth-aware design [77.34726150561087]
  We conduct a survey on the most relevant and recent advances in Deep Semantic in the context of vision for autonomous vehicles. Our main objective is to provide a comprehensive discussion on the main methods, advantages, limitations, results and challenges faced from each perspective.
  arXiv  Detail & Related papers  (2023-03-08T01:29:55Z)
• Temporal Perceiver: A General Architecture for Arbitrary Boundary Detection [48.33132632418303]
  Generic Boundary Detection (GBD) aims at locating general boundaries that divide videos into semantically coherent and taxonomy-free units. Previous research separately handle these different-level generic boundaries with specific designs of complicated deep networks from simple CNN to LSTM. We present Temporal Perceiver, a general architecture with Transformers, offering a unified solution to the detection of arbitrary generic boundaries.
  arXiv  Detail & Related papers  (2022-03-01T09:31:30Z)
• Robust Object Detection via Instance-Level Temporal Cycle Confusion [89.1027433760578]
  We study the effectiveness of auxiliary self-supervised tasks to improve the out-of-distribution generalization of object detectors. Inspired by the principle of maximum entropy, we introduce a novel self-supervised task, instance-level temporal cycle confusion (CycConf) For each object, the task is to find the most different object proposals in the adjacent frame in a video and then cycle back to itself for self-supervision.
  arXiv  Detail & Related papers  (2021-04-16T21:35:08Z)
• Self-supervised Human Detection and Segmentation via Multi-view Consensus [116.92405645348185]
  We propose a multi-camera framework in which geometric constraints are embedded in the form of multi-view consistency during training. We show that our approach outperforms state-of-the-art self-supervised person detection and segmentation techniques on images that visually depart from those of standard benchmarks.
  arXiv  Detail & Related papers  (2020-12-09T15:47:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.