Lessons from the Development of an Anomaly Detection Interface on the Mars Perseverance Rover using the ISHMAP Framework

• URL: http://arxiv.org/abs/2302.07187v1
• Date: Tue, 14 Feb 2023 16:55:32 GMT
• Title: Lessons from the Development of an Anomaly Detection Interface on the Mars Perseverance Rover using the ISHMAP Framework
• Authors: Austin P. Wright, Peter Nemere, Adrian Galvin, Duen Horng Chau, Scott Davidoff
• Abstract summary: We present the results of utilizing an alternative approach to machine learning based anomaly detection. We report on over 18 months of in-context user research and co-design to define the key problems NASA scientists face when looking to detect and interpret spectral anomalies. We develop a novel spectral anomaly detection toolkit for PIXL scientists that is highly accurate while maintaining strong transparency to scientific interpretation.
• Score: 8.353815643035498
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While anomaly detection stands among the most important and valuable problems across many scientific domains, anomaly detection research often focuses on AI methods that can lack the nuance and interpretability so critical to conducting scientific inquiry. In this application paper we present the results of utilizing an alternative approach that situates the mathematical framing of machine learning based anomaly detection within a participatory design framework. In a collaboration with NASA scientists working with the PIXL instrument studying Martian planetary geochemistry as a part of the search for extra-terrestrial life; we report on over 18 months of in-context user research and co-design to define the key problems NASA scientists face when looking to detect and interpret spectral anomalies. We address these problems and develop a novel spectral anomaly detection toolkit for PIXL scientists that is highly accurate while maintaining strong transparency to scientific interpretation. We also describe outcomes from a yearlong field deployment of the algorithm and associated interface. Finally we introduce a new design framework which we developed through the course of this collaboration for co-creating anomaly detection algorithms: Iterative Semantic Heuristic Modeling of Anomalous Phenomena (ISHMAP), which provides a process for scientists and researchers to produce natively interpretable anomaly detection models. This work showcases an example of successfully bridging methodologies from AI and HCI within a scientific domain, and provides a resource in ISHMAP which may be used by other researchers and practitioners looking to partner with other scientific teams to achieve better science through more effective and interpretable anomaly detection tools.

Related papers

• DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents [49.74065769505137]
  We introduce DISCOVERYWORLD, the first virtual environment for developing and benchmarking an agent's ability to perform complete cycles of novel scientific discovery. It includes 120 different challenge tasks spanning eight topics each with three levels of difficulty and several parametric variations. We find that strong baseline agents, that perform well in prior published environments, struggle on most DISCOVERYWORLD tasks.
  arXiv  Detail & Related papers  (2024-06-10T20:08:44Z)
• Progressing from Anomaly Detection to Automated Log Labeling and Pioneering Root Cause Analysis [53.24804865821692]
  This study introduces a taxonomy for log anomalies and explores automated data labeling to mitigate labeling challenges. The study envisions a future where root cause analysis follows anomaly detection, unraveling the underlying triggers of anomalies.
  arXiv  Detail & Related papers  (2023-12-22T15:04:20Z)
• Constructing Impactful Machine Learning Research for Astronomy: Best Practices for Researchers and Reviewers [0.0]
  Machine learning has rapidly become a tool of choice for the astronomical community. This paper provides a primer to the astronomical community on how to implement machine learning models and report their results.
  arXiv  Detail & Related papers  (2023-10-19T07:04:36Z)
• Searching for Novel Chemistry in Exoplanetary Atmospheres using Machine Learning for Anomaly Detection [1.8434042562191815]
  We advocate the application of machine learning (ML) techniques for anomaly (novelty) detection to exoplanet transit spectra. We demonstrate the feasibility of two popular anomaly detection methods on a large public database of synthetic spectra.
  arXiv  Detail & Related papers  (2023-08-15T07:19:54Z)
• A Diachronic Analysis of Paradigm Shifts in NLP Research: When, How, and Why? [84.46288849132634]
  We propose a systematic framework for analyzing the evolution of research topics in a scientific field using causal discovery and inference techniques. We define three variables to encompass diverse facets of the evolution of research topics within NLP. We utilize a causal discovery algorithm to unveil the causal connections among these variables using observational data.
  arXiv  Detail & Related papers  (2023-05-22T11:08:00Z)
• GFlowNets for AI-Driven Scientific Discovery [74.27219800878304]
  We present a new probabilistic machine learning framework called GFlowNets. GFlowNets can be applied in the modeling, hypotheses generation and experimental design stages of the experimental science loop. We argue that GFlowNets can become a valuable tool for AI-driven scientific discovery.
  arXiv  Detail & Related papers  (2023-02-01T17:29:43Z)
• Trust Your $\nabla$: Gradient-based Intervention Targeting for Causal Discovery [49.084423861263524]
  In this work, we propose a novel Gradient-based Intervention Targeting method, abbreviated GIT. GIT 'trusts' the gradient estimator of a gradient-based causal discovery framework to provide signals for the intervention acquisition function. We provide extensive experiments in simulated and real-world datasets and demonstrate that GIT performs on par with competitive baselines.
  arXiv  Detail & Related papers  (2022-11-24T17:04:45Z)
• Deep Learning for Time Series Anomaly Detection: A Survey [53.83593870825628]
  Time series anomaly detection has applications in a wide range of research fields and applications, including manufacturing and healthcare. The large size and complex patterns of time series have led researchers to develop specialised deep learning models for detecting anomalous patterns. This survey focuses on providing structured and comprehensive state-of-the-art time series anomaly detection models through the use of deep learning.
  arXiv  Detail & Related papers  (2022-11-09T22:40:22Z)
• Detecting and Diagnosing Terrestrial Gravitational-Wave Mimics Through Feature Learning [0.7388859384645262]
  We present a demonstration of a method that can detect and characterize emergent transient anomalies of massively complex systems. One of the prevalent issues limiting gravitational-wave discoveries is the noise artifacts of terrestrial origin. We show how a highly interpretable convolutional classifier can automatically learn to detect transient anomalies from auxiliary detector data.
  arXiv  Detail & Related papers  (2022-03-09T23:39:41Z)
• Applications of Generative Adversarial Networks in Anomaly Detection: A Systematic Literature Review [28.752089275446462]
  generative adversarial networks (GANs) have attracted a great deal of attention in anomaly detection research. In this paper, we present a systematic literature review of the applications of GANs in anomaly detection.
  arXiv  Detail & Related papers  (2021-10-22T21:48:48Z)
• Interactive Causal Structure Discovery in Earth System Sciences [6.788563219859884]
  Causal structure discovery (CSD) models are making inroads into several domains, including Earth system sciences. Their widespread adaptation is hampered by the fact that the resulting models often do not take into account the domain knowledge of the experts. We present a workflow that is required to take this knowledge into account and to apply CSD algorithms in Earth system sciences.
  arXiv  Detail & Related papers  (2021-07-01T09:23:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.