Anomaly Detection via Learning-Based Sequential Controlled Sensing

• URL: http://arxiv.org/abs/2312.00088v1
• Date: Thu, 30 Nov 2023 07:49:33 GMT
• Title: Anomaly Detection via Learning-Based Sequential Controlled Sensing
• Authors: Geethu Joseph, Chen Zhong, M. Cenk Gursoy, Senem Velipasalar, and Pramod K. Varshney
• Abstract summary: We address the problem of detecting anomalies among a set of binary processes via learning-based controlled sensing. To identify the anomalies, the decision-making agent is allowed to observe a subset of the processes at each time instant. Our objective is to design a sequential selection policy that dynamically determines which processes to observe at each time.
• Score: 25.282033825977827
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we address the problem of detecting anomalies among a given set of binary processes via learning-based controlled sensing. Each process is parameterized by a binary random variable indicating whether the process is anomalous. To identify the anomalies, the decision-making agent is allowed to observe a subset of the processes at each time instant. Also, probing each process has an associated cost. Our objective is to design a sequential selection policy that dynamically determines which processes to observe at each time with the goal to minimize the delay in making the decision and the total sensing cost. We cast this problem as a sequential hypothesis testing problem within the framework of Markov decision processes. This formulation utilizes both a Bayesian log-likelihood ratio-based reward and an entropy-based reward. The problem is then solved using two approaches: 1) a deep reinforcement learning-based approach where we design both deep Q-learning and policy gradient actor-critic algorithms; and 2) a deep active inference-based approach. Using numerical experiments, we demonstrate the efficacy of our algorithms and show that our algorithms adapt to any unknown statistical dependence pattern of the processes.

Related papers

• Provably Efficient UCB-type Algorithms For Learning Predictive State Representations [55.00359893021461]
  The sequential decision-making problem is statistically learnable if it admits a low-rank structure modeled by predictive state representations (PSRs) This paper proposes the first known UCB-type approach for PSRs, featuring a novel bonus term that upper bounds the total variation distance between the estimated and true models. In contrast to existing approaches for PSRs, our UCB-type algorithms enjoy computational tractability, last-iterate guaranteed near-optimal policy, and guaranteed model accuracy.
  arXiv  Detail & Related papers  (2023-07-01T18:35:21Z)
• Interactive System-wise Anomaly Detection [66.3766756452743]
  Anomaly detection plays a fundamental role in various applications. It is challenging for existing methods to handle the scenarios where the instances are systems whose characteristics are not readily observed as data. We develop an end-to-end approach which includes an encoder-decoder module that learns system embeddings.
  arXiv  Detail & Related papers  (2023-04-21T02:20:24Z)
• Temporal Detection of Anomalies via Actor-Critic Based Controlled Sensing [31.841289319809807]
  We address the problem of monitoring a set of binary processes and generating an alert when the number of anomalies among them exceeds a threshold. For this, the decision-maker selects and probes a subset of the processes to obtain noisy estimates of their states. Using the posterior probability, we construct a Markov decision process and solve it using deep actor-critic reinforcement learning.
  arXiv  Detail & Related papers  (2022-01-03T20:59:40Z)
• Scalable and Decentralized Algorithms for Anomaly Detection via Learning-Based Controlled Sensing [40.14838268469627]
  We develop an anomaly detection algorithm that chooses the processes to be observed at a given time instant. The objective of the detection algorithm is to identify the anomalies with an accuracy exceeding the desired value.
  arXiv  Detail & Related papers  (2021-12-08T11:20:36Z)
• Anomaly Detection via Controlled Sensing and Deep Active Inference [43.07302992747749]
  In this paper, we address the anomaly detection problem where the objective is to find the anomalous processes among a given set of processes. We develop a sequential selection algorithm that decides which processes to be probed at every instant to detect the anomalies. Our algorithm is based on active inference which is a general framework to make sequential decisions in order to maximize the notion of free energy.
  arXiv  Detail & Related papers  (2021-05-12T17:54:02Z)
• A Scalable Algorithm for Anomaly Detection via Learning-Based Controlled Sensing [37.78306297797]
  We develop an anomaly detection algorithm that chooses the process to be observed at a given time instant. The objective of the detection algorithm is to arrive at a decision with an accuracy exceeding a desired value. Unlike prior work on this topic that has exponential complexity in the number of processes, our algorithm has computational and memory requirements that are both in the number of processes.
  arXiv  Detail & Related papers  (2021-05-12T17:46:01Z)
• Adaptive Sampling for Best Policy Identification in Markov Decision Processes [79.4957965474334]
  We investigate the problem of best-policy identification in discounted Markov Decision (MDPs) when the learner has access to a generative model. The advantages of state-of-the-art algorithms are discussed and illustrated.
  arXiv  Detail & Related papers  (2020-09-28T15:22:24Z)
• Run2Survive: A Decision-theoretic Approach to Algorithm Selection based on Survival Analysis [75.64261155172856]
  survival analysis (SA) naturally supports censored data and offers appropriate ways to use such data for learning distributional models of algorithm runtime. We leverage such models as a basis of a sophisticated decision-theoretic approach to algorithm selection, which we dub Run2Survive. In an extensive experimental study with the standard benchmark ASlib, our approach is shown to be highly competitive and in many cases even superior to state-of-the-art AS approaches.
  arXiv  Detail & Related papers  (2020-07-06T15:20:17Z)
• Anomaly Detection Under Controlled Sensing Using Actor-Critic Reinforcement Learning [31.841289319809807]
  We consider the problem of detecting anomalies among a given set of processes using noisy binary sensor measurements. The noiseless sensor measurement corresponding to a normal process is 0, and the measurement is 1 if the process is anomalous. We propose a sequential sensor selection policy that dynamically determines which processes to observe at each time and when to terminate the detection algorithm.
  arXiv  Detail & Related papers  (2020-05-26T22:53:17Z)
• Active Model Estimation in Markov Decision Processes [108.46146218973189]
  We study the problem of efficient exploration in order to learn an accurate model of an environment, modeled as a Markov decision process (MDP) We show that our Markov-based algorithm outperforms both our original algorithm and the maximum entropy algorithm in the small sample regime.
  arXiv  Detail & Related papers  (2020-03-06T16:17:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.