Fine-Grained Off-Road Semantic Segmentation and Mapping via Contrastive
Learning
- URL: http://arxiv.org/abs/2103.03651v1
- Date: Fri, 5 Mar 2021 13:23:24 GMT
- Title: Fine-Grained Off-Road Semantic Segmentation and Mapping via Contrastive
Learning
- Authors: Biao Gao, Shaochi Hu, Xijun Zhao, Huijing Zhao
- Abstract summary: Road detection or traversability analysis has been a key technique for a mobile robot to traverse complex off-road scenes.
understanding scenes with fine-grained labels are needed for off-road robots, as scenes are very diverse.
This research proposes a contrastive learning based method to achieve meaningful scene understanding for a robot to traverse off-road.
- Score: 7.965964259208489
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Road detection or traversability analysis has been a key technique for a
mobile robot to traverse complex off-road scenes. The problem has been mainly
formulated in early works as a binary classification one, e.g. associating
pixels with road or non-road labels. Whereas understanding scenes with
fine-grained labels are needed for off-road robots, as scenes are very diverse,
and the various mechanical performance of off-road robots may lead to different
definitions of safe regions to traverse. How to define and annotate
fine-grained labels to achieve meaningful scene understanding for a robot to
traverse off-road is still an open question. This research proposes a
contrastive learning based method. With a set of human-annotated anchor
patches, a feature representation is learned to discriminate regions with
different traversability, a method of fine-grained semantic segmentation and
mapping is subsequently developed for off-road scene understanding. Experiments
are conducted on a dataset of three driving segments that represent very
diverse off-road scenes. An anchor accuracy of 89.8% is achieved by evaluating
the matching with human-annotated image patches in cross-scene validation.
Examined by associated 3D LiDAR data, the fine-grained segments of visual
images are demonstrated to have different levels of toughness and terrain
elevation, which represents their semantical meaningfulness. The resultant maps
contain both fine-grained labels and confidence values, providing rich
information to support a robot traversing complex off-road scenes.
Related papers
- Mapping High-level Semantic Regions in Indoor Environments without
Object Recognition [50.624970503498226]
The present work proposes a method for semantic region mapping via embodied navigation in indoor environments.
To enable region identification, the method uses a vision-to-language model to provide scene information for mapping.
By projecting egocentric scene understanding into the global frame, the proposed method generates a semantic map as a distribution over possible region labels at each location.
arXiv Detail & Related papers (2024-03-11T18:09:50Z) - CarPatch: A Synthetic Benchmark for Radiance Field Evaluation on Vehicle
Components [77.33782775860028]
We introduce CarPatch, a novel synthetic benchmark of vehicles.
In addition to a set of images annotated with their intrinsic and extrinsic camera parameters, the corresponding depth maps and semantic segmentation masks have been generated for each view.
Global and part-based metrics have been defined and used to evaluate, compare, and better characterize some state-of-the-art techniques.
arXiv Detail & Related papers (2023-07-24T11:59:07Z) - OpenLane-V2: A Topology Reasoning Benchmark for Unified 3D HD Mapping [84.65114565766596]
We present OpenLane-V2, the first dataset on topology reasoning for traffic scene structure.
OpenLane-V2 consists of 2,000 annotated road scenes that describe traffic elements and their correlation to the lanes.
We evaluate various state-of-the-art methods, and present their quantitative and qualitative results on OpenLane-V2 to indicate future avenues for investigating topology reasoning in traffic scenes.
arXiv Detail & Related papers (2023-04-20T16:31:22Z) - Graph-based Topology Reasoning for Driving Scenes [102.35885039110057]
We present TopoNet, the first end-to-end framework capable of abstracting traffic knowledge beyond conventional perception tasks.
We evaluate TopoNet on the challenging scene understanding benchmark, OpenLane-V2.
arXiv Detail & Related papers (2023-04-11T15:23:29Z) - Exploring Contextual Representation and Multi-Modality for End-to-End
Autonomous Driving [58.879758550901364]
Recent perception systems enhance spatial understanding with sensor fusion but often lack full environmental context.
We introduce a framework that integrates three cameras to emulate the human field of view, coupled with top-down bird-eye-view semantic data to enhance contextual representation.
Our method achieves displacement error by 0.67m in open-loop settings, surpassing current methods by 6.9% on the nuScenes dataset.
arXiv Detail & Related papers (2022-10-13T05:56:20Z) - An Active and Contrastive Learning Framework for Fine-Grained Off-Road
Semantic Segmentation [7.035838394813961]
Off-road semantic segmentation with fine-grained labels is necessary for autonomous vehicles to understand driving scenes.
Fine-grained semantic segmentation in off-road scenes usually has no unified category definition due to ambiguous nature environments.
This research proposes an active and contrastive learning-based method that does not rely on pixel-wise labels.
arXiv Detail & Related papers (2022-02-18T03:16:31Z) - Navigation-Oriented Scene Understanding for Robotic Autonomy: Learning
to Segment Driveability in Egocentric Images [25.350677396144075]
This work tackles scene understanding for outdoor robotic navigation, solely relying on images captured by an on-board camera.
We segment egocentric images directly in terms of how a robot can navigate in them, and tailor the learning problem to an autonomous navigation task.
We present a generic and scalable affordance-based definition consisting of 3 driveability levels which can be applied to arbitrary scenes.
arXiv Detail & Related papers (2021-09-15T12:25:56Z) - OFFSEG: A Semantic Segmentation Framework For Off-Road Driving [6.845371503461449]
We propose a framework for off-road semantic segmentation called as OFFSEG.
Off-road semantic segmentation is challenging due to the presence of uneven terrains, unstructured class boundaries, irregular features and strong textures.
arXiv Detail & Related papers (2021-03-23T09:45:41Z) - An Image-based Approach of Task-driven Driving Scene Categorization [7.291979964739049]
This paper proposes a method of task-driven driving scene categorization using weakly supervised data.
A measure is learned to discriminate the scenes of different semantic attributes via contrastive learning.
The results of semantic scene similarity learning and driving scene categorization are extensively studied.
arXiv Detail & Related papers (2021-03-10T08:23:36Z) - Scribble-based Weakly Supervised Deep Learning for Road Surface
Extraction from Remote Sensing Images [7.1577508803778045]
We propose a scribble-based weakly supervised road surface extraction method named ScRoadExtractor.
To propagate semantic information from sparse scribbles to unlabeled pixels, we introduce a road label propagation algorithm.
The proposal masks generated from the road label propagation algorithm are utilized to train a dual-branch encoder-decoder network.
arXiv Detail & Related papers (2020-10-25T12:40:30Z) - BoMuDANet: Unsupervised Adaptation for Visual Scene Understanding in
Unstructured Driving Environments [54.22535063244038]
We present an unsupervised adaptation approach for visual scene understanding in unstructured traffic environments.
Our method is designed for unstructured real-world scenarios with dense and heterogeneous traffic consisting of cars, trucks, two-and three-wheelers, and pedestrians.
arXiv Detail & Related papers (2020-09-22T08:25:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.