Related papers: What you see is what you get: Experience ranking with deep neural dataset-to-dataset similarity for topological localisation

What you see is what you get: Experience ranking with deep neural dataset-to-dataset similarity for topological localisation

URL: http://arxiv.org/abs/2310.13622v1
Date: Fri, 20 Oct 2023 16:13:21 GMT
Title: What you see is what you get: Experience ranking with deep neural dataset-to-dataset similarity for topological localisation
Authors: Matthew Gadd, Benjamin Ramtoula, Daniele De Martini, Paul Newman
Abstract summary: We propose applying the recently developed Visual DNA as a highly scalable tool for comparing datasets of images. In the case of localisation, important dataset differences impacting performance are modes of appearance change, including weather, lighting, and season. We find that differences in these statistics correlate to performance when localising using a past experience with the same appearance gap.
Score: 19.000718685399935
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Recalling the most relevant visual memories for localisation or understanding a priori the likely outcome of localisation effort against a particular visual memory is useful for efficient and robust visual navigation. Solutions to this problem should be divorced from performance appraisal against ground truth - as this is not available at run-time - and should ideally be based on generalisable environmental observations. For this, we propose applying the recently developed Visual DNA as a highly scalable tool for comparing datasets of images - in this work, sequences of map and live experiences. In the case of localisation, important dataset differences impacting performance are modes of appearance change, including weather, lighting, and season. Specifically, for any deep architecture which is used for place recognition by matching feature volumes at a particular layer, we use distribution measures to compare neuron-wise activation statistics between live images and multiple previously recorded past experiences, with a potentially large seasonal (winter/summer) or time of day (day/night) shift. We find that differences in these statistics correlate to performance when localising using a past experience with the same appearance gap. We validate our approach over the Nordland cross-season dataset as well as data from Oxford's University Parks with lighting and mild seasonal change, showing excellent ability of our system to rank actual localisation performance across candidate experiences.

Related papers

Diffusion Models as Data Mining Tools [87.77999285241219]
This paper demonstrates how to use generative models trained for image synthesis as tools for visual data mining. We show that after finetuning conditional diffusion models to synthesize images from a specific dataset, we can use these models to define a typicality measure. This measure assesses how typical visual elements are for different data labels, such as geographic location, time stamps, semantic labels, or even the presence of a disease.
arXiv Detail & Related papers (2024-07-20T17:14:31Z)
Localization Is All You Evaluate: Data Leakage in Online Mapping Datasets and How to Fix It [2.1665407462280446]
State-of-the-art methods are trained predominantly using nuScenes and Argoverse 2 datasets. Over $80$% of nuScenes and $40$% of Argoverse 2 validation and test samples are less than $5$ m from a training sample. We propose geographically disjoint data splits to reveal the true performance in unseen environments.
arXiv Detail & Related papers (2023-12-11T14:43:23Z)
Long-Term Invariant Local Features via Implicit Cross-Domain Correspondences [79.21515035128832]
We conduct a thorough analysis of the performance of current state-of-the-art feature extraction networks under various domain changes. We propose a novel data-centric method, Implicit Cross-Domain Correspondences (iCDC) iCDC represents the same environment with multiple Neural Radiance Fields, each fitting the scene under individual visual domains.
arXiv Detail & Related papers (2023-11-06T18:53:01Z)
Concept Drift and Long-Tailed Distribution in Fine-Grained Visual Categorization: Benchmark and Method [84.68818879525568]
We present a Concept Drift and Long-Tailed Distribution dataset. The characteristics of instances tend to vary with time and exhibit a long-tailed distribution. We propose a feature recombination framework to address the learning challenges associated with CDLT.
arXiv Detail & Related papers (2023-06-04T12:42:45Z)
4Seasons: Benchmarking Visual SLAM and Long-Term Localization for Autonomous Driving in Challenging Conditions [54.59279160621111]
We present a novel visual SLAM and long-term localization benchmark for autonomous driving in challenging conditions based on the large-scale 4Seasons dataset. The proposed benchmark provides drastic appearance variations caused by seasonal changes and diverse weather and illumination conditions. We introduce a new unified benchmark for jointly evaluating visual odometry, global place recognition, and map-based visual localization performance.
arXiv Detail & Related papers (2022-12-31T13:52:36Z)
Location-aware Adaptive Normalization: A Deep Learning Approach For Wildfire Danger Forecasting [17.25189382307337]
This paper proposes a 2D/3D two-branch convolutional neural network (CNN) with a Location-aware Adaptive Normalization layer (LOAN) Using LOAN as a building block, we can modulate the dynamic features conditional on their geographical locations. Results show that location-aware adaptive feature normalization is a promising technique to learn the relation between dynamic variables and their geographic locations.
arXiv Detail & Related papers (2022-12-16T00:32:38Z)
A Closer Look at Debiased Temporal Sentence Grounding in Videos: Dataset, Metric, and Approach [53.727460222955266]
Temporal Sentence Grounding in Videos (TSGV) aims to ground a natural language sentence in an untrimmed video. Recent studies have found that current benchmark datasets may have obvious moment annotation biases. We introduce a new evaluation metric "dR@n,IoU@m" that discounts the basic recall scores to alleviate the inflating evaluation caused by biased datasets.
arXiv Detail & Related papers (2022-03-10T08:58:18Z)
CrowdDriven: A New Challenging Dataset for Outdoor Visual Localization [44.97567243883994]
We propose a new benchmark for visual localization in outdoor scenes using crowd-sourced data. We show that our dataset is very challenging, with all evaluated methods failing on its hardest parts. As part of the dataset release, we provide the tooling used to generate it, enabling efficient and effective 2D correspondence annotation.
arXiv Detail & Related papers (2021-09-09T19:25:48Z)
Domain-invariant Similarity Activation Map Contrastive Learning for Retrieval-based Long-term Visual Localization [30.203072945001136]
In this work, a general architecture is first formulated probabilistically to extract domain invariant feature through multi-domain image translation. And then a novel gradient-weighted similarity activation mapping loss (Grad-SAM) is incorporated for finer localization with high accuracy. Extensive experiments have been conducted to validate the effectiveness of the proposed approach on the CMUSeasons dataset. Our performance is on par with or even outperforms the state-of-the-art image-based localization baselines in medium or high precision.
arXiv Detail & Related papers (2020-09-16T14:43:22Z)
Domain Adaptation of Learned Features for Visual Localization [60.6817896667435]
We tackle the problem of visual localization under changing conditions, such as time of day, weather, and seasons. Recent learned local features based on deep neural networks have shown superior performance over classical hand-crafted local features. We present a novel and practical approach, where only a few examples are needed to reduce the domain gap.
arXiv Detail & Related papers (2020-08-21T05:17:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.