Temporal Detection of Anomalies via Actor-Critic Based Controlled Sensing

• URL: http://arxiv.org/abs/2201.00879v2
• Date: Fri, 16 Jun 2023 11:51:06 GMT
• Title: Temporal Detection of Anomalies via Actor-Critic Based Controlled Sensing
• Authors: Geethu Joseph, M. Cenk Gursoy, Pramod K. Varshney
• Abstract summary: We address the problem of monitoring a set of binary processes and generating an alert when the number of anomalies among them exceeds a threshold. For this, the decision-maker selects and probes a subset of the processes to obtain noisy estimates of their states. Using the posterior probability, we construct a Markov decision process and solve it using deep actor-critic reinforcement learning.
• Score: 31.841289319809807
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We address the problem of monitoring a set of binary stochastic processes and generating an alert when the number of anomalies among them exceeds a threshold. For this, the decision-maker selects and probes a subset of the processes to obtain noisy estimates of their states (normal or anomalous). Based on the received observations, the decisionmaker first determines whether to declare that the number of anomalies has exceeded the threshold or to continue taking observations. When the decision is to continue, it then decides whether to collect observations at the next time instant or defer it to a later time. If it chooses to collect observations, it further determines the subset of processes to be probed. To devise this three-step sequential decision-making process, we use a Bayesian formulation wherein we learn the posterior probability on the states of the processes. Using the posterior probability, we construct a Markov decision process and solve it using deep actor-critic reinforcement learning. Via numerical experiments, we demonstrate the superior performance of our algorithm compared to the traditional model-based algorithms.

Related papers

• Anomaly Detection via Learning-Based Sequential Controlled Sensing [25.282033825977827]
  We address the problem of detecting anomalies among a set of binary processes via learning-based controlled sensing. To identify the anomalies, the decision-making agent is allowed to observe a subset of the processes at each time instant. Our objective is to design a sequential selection policy that dynamically determines which processes to observe at each time.
  arXiv  Detail & Related papers  (2023-11-30T07:49:33Z)
• Shortcomings of Top-Down Randomization-Based Sanity Checks for Evaluations of Deep Neural Network Explanations [67.40641255908443]
  We identify limitations of model-randomization-based sanity checks for the purpose of evaluating explanations. Top-down model randomization preserves scales of forward pass activations with high probability.
  arXiv  Detail & Related papers  (2022-11-22T18:52:38Z)
• Scalable and Decentralized Algorithms for Anomaly Detection via Learning-Based Controlled Sensing [40.14838268469627]
  We develop an anomaly detection algorithm that chooses the processes to be observed at a given time instant. The objective of the detection algorithm is to identify the anomalies with an accuracy exceeding the desired value.
  arXiv  Detail & Related papers  (2021-12-08T11:20:36Z)
• A2Log: Attentive Augmented Log Anomaly Detection [53.06341151551106]
  Anomaly detection becomes increasingly important for the dependability and serviceability of IT services. Existing unsupervised methods need anomaly examples to obtain a suitable decision boundary. We develop A2Log, which is an unsupervised anomaly detection method consisting of two steps: Anomaly scoring and anomaly decision.
  arXiv  Detail & Related papers  (2021-09-20T13:40:21Z)
• Machine Learning for Online Algorithm Selection under Censored Feedback [71.6879432974126]
  In online algorithm selection (OAS), instances of an algorithmic problem class are presented to an agent one after another, and the agent has to quickly select a presumably best algorithm from a fixed set of candidate algorithms. For decision problems such as satisfiability (SAT), quality typically refers to the algorithm's runtime. In this work, we revisit multi-armed bandit algorithms for OAS and discuss their capability of dealing with the problem. We adapt them towards runtime-oriented losses, allowing for partially censored data while keeping a space- and time-complexity independent of the time horizon.
  arXiv  Detail & Related papers  (2021-09-13T18:10:52Z)
• Navigating to the Best Policy in Markov Decision Processes [68.8204255655161]
  We investigate the active pure exploration problem in Markov Decision Processes. Agent sequentially selects actions and, from the resulting system trajectory, aims at the best as fast as possible.
  arXiv  Detail & Related papers  (2021-06-05T09:16:28Z)
• Anomaly Detection via Controlled Sensing and Deep Active Inference [43.07302992747749]
  In this paper, we address the anomaly detection problem where the objective is to find the anomalous processes among a given set of processes. We develop a sequential selection algorithm that decides which processes to be probed at every instant to detect the anomalies. Our algorithm is based on active inference which is a general framework to make sequential decisions in order to maximize the notion of free energy.
  arXiv  Detail & Related papers  (2021-05-12T17:54:02Z)
• A Scalable Algorithm for Anomaly Detection via Learning-Based Controlled Sensing [37.78306297797]
  We develop an anomaly detection algorithm that chooses the process to be observed at a given time instant. The objective of the detection algorithm is to arrive at a decision with an accuracy exceeding a desired value. Unlike prior work on this topic that has exponential complexity in the number of processes, our algorithm has computational and memory requirements that are both in the number of processes.
  arXiv  Detail & Related papers  (2021-05-12T17:46:01Z)
• Anomaly Detection Under Controlled Sensing Using Actor-Critic Reinforcement Learning [31.841289319809807]
  We consider the problem of detecting anomalies among a given set of processes using noisy binary sensor measurements. The noiseless sensor measurement corresponding to a normal process is 0, and the measurement is 1 if the process is anomalous. We propose a sequential sensor selection policy that dynamically determines which processes to observe at each time and when to terminate the detection algorithm.
  arXiv  Detail & Related papers  (2020-05-26T22:53:17Z)
• Active Model Estimation in Markov Decision Processes [108.46146218973189]
  We study the problem of efficient exploration in order to learn an accurate model of an environment, modeled as a Markov decision process (MDP) We show that our Markov-based algorithm outperforms both our original algorithm and the maximum entropy algorithm in the small sample regime.
  arXiv  Detail & Related papers  (2020-03-06T16:17:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.