Attention-Driven Hierarchical Reinforcement Learning with Particle Filtering for Source Localization in Dynamic Fields

• URL: http://arxiv.org/abs/2501.13084v1
• Date: Wed, 22 Jan 2025 18:45:29 GMT
• Title: Attention-Driven Hierarchical Reinforcement Learning with Particle Filtering for Source Localization in Dynamic Fields
• Authors: Yiwei Shi, Mengyue Yang, Qi Zhang, Weinan Zhang, Cunjia Liu, Weiru Liu,
• Abstract summary: We propose a hierarchical framework that integrates Bayesian inference and reinforcement learning. Our results highlight the framework's potential for broad applications in dynamic field estimation tasks.
• Score: 24.30636951062104
• License:
• Abstract: In many real-world scenarios, such as gas leak detection or environmental pollutant tracking, solving the Inverse Source Localization and Characterization problem involves navigating complex, dynamic fields with sparse and noisy observations. Traditional methods face significant challenges, including partial observability, temporal and spatial dynamics, out-of-distribution generalization, and reward sparsity. To address these issues, we propose a hierarchical framework that integrates Bayesian inference and reinforcement learning. The framework leverages an attention-enhanced particle filtering mechanism for efficient and accurate belief updates, and incorporates two complementary execution strategies: Attention Particle Filtering Planning and Attention Particle Filtering Reinforcement Learning. These approaches optimize exploration and adaptation under uncertainty. Theoretical analysis proves the convergence of the attention-enhanced particle filter, while extensive experiments across diverse scenarios validate the framework's superior accuracy, adaptability, and computational efficiency. Our results highlight the framework's potential for broad applications in dynamic field estimation tasks.

Related papers

• Beyond Prior Limits: Addressing Distribution Misalignment in Particle Filtering [22.722446981864046]
  Particle is a fundamental tool in state estimation for dynamic systems, but its effectiveness is often limited by the constraints of prior distribution. We propose the Diffusion-Enhanced Particle Framework, which includes three key innovations: adaptive diffusion through particles, entropy-driven regularisation to prevent weight collapse, and kernel-based perturbations for dynamic support expansion.
  arXiv  Detail & Related papers  (2025-01-30T17:11:34Z)
• Constrained Optimization of Charged Particle Tracking with Multi-Agent Reinforcement Learning [0.0]
  We propose a multi-agent reinforcement learning approach with assignment constraints for reconstructing particle tracks in pixelated particle detectors. Our approach optimises collaboratively a parametrized policy, functioning as a to a multidimensional assignment problem. We empirically show on simulated data, generated for a particle detector developed for proton imaging, the effectiveness of our approach, compared to multiple single- and multi-agent baselines.
  arXiv  Detail & Related papers  (2025-01-09T09:59:42Z)
• Adapting Physics-Informed Neural Networks for Bifurcation Detection in Ecological Migration Models [0.16442870218029523]
  In this study, we explore the application of Physics-Informed Neural Networks (PINNs) to the analysis of bifurcation phenomena in ecological migration models. By integrating the fundamental principles of diffusion-advection-reaction equations with deep learning techniques, we address the complexities of species migration dynamics.
  arXiv  Detail & Related papers  (2024-09-01T08:00:31Z)
• Exploring End-to-end Differentiable Neural Charged Particle Tracking -- A Loss Landscape Perspective [0.0]
  We propose an E2E differentiable decision-focused learning scheme for particle tracking. We show that differentiable variations of discrete assignment operations allows for efficient network optimization. We argue that E2E differentiability provides, besides the general availability of gradient information, an important tool for robust particle tracking to mitigate prediction instabilities.
  arXiv  Detail & Related papers  (2024-07-18T11:42:58Z)
• Spectral Decomposition Representation for Reinforcement Learning [100.0424588013549]
  We propose an alternative spectral method, Spectral Decomposition Representation (SPEDER), that extracts a state-action abstraction from the dynamics without inducing spurious dependence on the data collection policy. A theoretical analysis establishes the sample efficiency of the proposed algorithm in both the online and offline settings. An experimental investigation demonstrates superior performance over current state-of-the-art algorithms across several benchmarks.
  arXiv  Detail & Related papers  (2022-08-19T19:01:30Z)
• AntPivot: Livestream Highlight Detection via Hierarchical Attention Mechanism [64.70568612993416]
  We formulate a new task Livestream Highlight Detection, discuss and analyze the difficulties listed above and propose a novel architecture AntPivot to solve this problem. We construct a fully-annotated dataset AntHighlight to instantiate this task and evaluate the performance of our model.
  arXiv  Detail & Related papers  (2022-06-10T05:58:11Z)
• Learnable Multi-level Frequency Decomposition and Hierarchical Attention Mechanism for Generalized Face Presentation Attack Detection [7.324459578044212]
  Face presentation attack detection (PAD) is attracting a lot of attention and playing a key role in securing face recognition systems. We propose a dual-stream convolution neural networks (CNNs) framework to deal with unseen scenarios. We successfully prove the design of our proposed PAD solution in a step-wise ablation study.
  arXiv  Detail & Related papers  (2021-09-16T13:06:43Z)
• Focus of Attention Improves Information Transfer in Visual Features [80.22965663534556]
  This paper focuses on unsupervised learning for transferring visual information in a truly online setting. The computation of the entropy terms is carried out by a temporal process which yields online estimation of the entropy terms. In order to better structure the input probability distribution, we use a human-like focus of attention model.
  arXiv  Detail & Related papers  (2020-06-16T15:07:25Z)
• When is Particle Filtering Efficient for Planning in Partially Observed Linear Dynamical Systems? [60.703816720093016]
  This paper initiates a study on the efficiency of particle filtering for sequential planning. We are able to bound the number of particles needed so that the long-run reward of the policy based on particle filtering is close to that based on exact inference. We believe this technique can be useful in other sequential decision-making problems.
  arXiv  Detail & Related papers  (2020-06-10T17:43:43Z)
• Robust Reinforcement Learning with Wasserstein Constraint [49.86490922809473]
  We show the existence of optimal robust policies, provide a sensitivity analysis for the perturbations, and then design a novel robust learning algorithm. The effectiveness of the proposed algorithm is verified in the Cart-Pole environment.
  arXiv  Detail & Related papers  (2020-06-01T13:48:59Z)
• Discrete Action On-Policy Learning with Action-Value Critic [72.20609919995086]
  Reinforcement learning (RL) in discrete action space is ubiquitous in real-world applications, but its complexity grows exponentially with the action-space dimension. We construct a critic to estimate action-value functions, apply it on correlated actions, and combine these critic estimated action values to control the variance of gradient estimation. These efforts result in a new discrete action on-policy RL algorithm that empirically outperforms related on-policy algorithms relying on variance control techniques.
  arXiv  Detail & Related papers  (2020-02-10T04:23:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.