Evaluation Challenges for Geospatial ML

• URL: http://arxiv.org/abs/2303.18087v1
• Date: Fri, 31 Mar 2023 14:24:06 GMT
• Title: Evaluation Challenges for Geospatial ML
• Authors: Esther Rolf
• Abstract summary: Geospatial machine learning models and maps are increasingly used for downstream analyses in science and policy. The correct way to measure performance of spatial machine learning outputs has been a topic of debate. This paper delineates unique challenges of model evaluation for geospatial machine learning with global or remotely sensed datasets.
• Score: 5.576083740549639
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As geospatial machine learning models and maps derived from their predictions are increasingly used for downstream analyses in science and policy, it is imperative to evaluate their accuracy and applicability. Geospatial machine learning has key distinctions from other learning paradigms, and as such, the correct way to measure performance of spatial machine learning outputs has been a topic of debate. In this paper, I delineate unique challenges of model evaluation for geospatial machine learning with global or remotely sensed datasets, culminating in concrete takeaways to improve evaluations of geospatial model performance.

Related papers

• An Ensemble Framework for Explainable Geospatial Machine Learning Models [16.010404125829876]
  We introduce an integrated framework that merges local spatial weighting scheme, Explainable Artificial Intelligence (XAI) and cutting-edge machine learning technologies. This framework is verified to enhance the interpretability and accuracy of predictions in both geographic regression and classification. It significantly boosts prediction precision, offering a novel approach to understanding spatial phenomena.
  arXiv  Detail & Related papers  (2024-03-05T21:12:10Z)
• Better, Not Just More: Data-Centric Machine Learning for Earth Observation [16.729827218159038]
  We argue that a shift from a model-centric view to a complementary data-centric perspective is necessary for further improvements in accuracy, generalization ability, and real impact on end-user applications. This work presents a definition as well as a precise categorization and overview of automated data-centric learning approaches for geospatial data.
  arXiv  Detail & Related papers  (2023-12-08T19:24:05Z)
• Challenges in data-based geospatial modeling for environmental research and practice [19.316860936437823]
  Data-based geospatial modelling using machine learning (ML) has gained popularity in environmental research. This survey reviews common nuances in geospatial modelling, such as imbalanced data, spatial autocorrelation, prediction errors, model generalisation, domain specificity, and uncertainty estimation.
  arXiv  Detail & Related papers  (2023-11-18T12:30:49Z)
• Explainable GeoAI: Can saliency maps help interpret artificial intelligence's learning process? An empirical study on natural feature detection [4.52308938611108]
  This paper compares popular saliency map generation techniques and their strengths and weaknesses in interpreting GeoAI and deep learning models' reasoning behaviors. The experiments used two GeoAI-ready datasets to demonstrate the generalizability of the research findings.
  arXiv  Detail & Related papers  (2023-03-16T21:37:29Z)
• Generalization Properties of Retrieval-based Models [50.35325326050263]
  Retrieval-based machine learning methods have enjoyed success on a wide range of problems. Despite growing literature showcasing the promise of these models, the theoretical underpinning for such models remains underexplored. We present a formal treatment of retrieval-based models to characterize their generalization ability.
  arXiv  Detail & Related papers  (2022-10-06T00:33:01Z)
• Spatial machine-learning model diagnostics: a model-agnostic distance-based approach [91.62936410696409]
  This contribution proposes spatial prediction error profiles (SPEPs) and spatial variable importance profiles (SVIPs) as novel model-agnostic assessment and interpretation tools. The SPEPs and SVIPs of geostatistical methods, linear models, random forest, and hybrid algorithms show striking differences and also relevant similarities. The novel diagnostic tools enrich the toolkit of spatial data science, and may improve ML model interpretation, selection, and design.
  arXiv  Detail & Related papers  (2021-11-13T01:50:36Z)
• ALT-MAS: A Data-Efficient Framework for Active Testing of Machine Learning Algorithms [58.684954492439424]
  We propose a novel framework to efficiently test a machine learning model using only a small amount of labeled test data. The idea is to estimate the metrics of interest for a model-under-test using Bayesian neural network (BNN)
  arXiv  Detail & Related papers  (2021-04-11T12:14:04Z)
• Geostatistical Learning: Challenges and Opportunities [0.0]
  We introduce the geostatistical (transfer) learning problem, and illustrate the challenges of learning from geospatial data. Experiments with synthetic Gaussian process data as well as with real data from geophysical surveys in New Zealand indicate that none of the methods are adequate for model selection in a geospatial context.
  arXiv  Detail & Related papers  (2021-02-17T14:33:15Z)
• Surface Warping Incorporating Machine Learning Assisted Domain Likelihood Estimation: A New Paradigm in Mine Geology Modelling and Automation [68.8204255655161]
  A Bayesian warping technique has been proposed to reshape modeled surfaces based on geochemical and spatial constraints imposed by newly acquired blasthole data. This paper focuses on incorporating machine learning in this warping framework to make the likelihood generalizable. Its foundation is laid by a Bayesian computation in which the geological domain likelihood given the chemistry, p(g|c) plays a similar role to p(y(c)|g.
  arXiv  Detail & Related papers  (2021-02-15T10:37:52Z)
• A User's Guide to Calibrating Robotics Simulators [54.85241102329546]
  This paper proposes a set of benchmarks and a framework for the study of various algorithms aimed to transfer models and policies learnt in simulation to the real world. We conduct experiments on a wide range of well known simulated environments to characterize and offer insights into the performance of different algorithms. Our analysis can be useful for practitioners working in this area and can help make informed choices about the behavior and main properties of sim-to-real algorithms.
  arXiv  Detail & Related papers  (2020-11-17T22:24:26Z)
• A Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence [48.67755344239951]
  We provide a comprehensive survey, and propose a new taxonomy for localization and mapping using deep learning. A wide range of topics are covered, from learning odometry estimation, mapping, to global localization and simultaneous localization and mapping. It is our hope that this work can connect emerging works from robotics, computer vision and machine learning communities.
  arXiv  Detail & Related papers  (2020-06-22T19:01:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.