State Estimation Using Particle Filtering in Adaptive Machine Learning Methods: Integrating Q-Learning and NEAT Algorithms with Noisy Radar Measurements

• URL: http://arxiv.org/abs/2504.07393v1
• Date: Thu, 10 Apr 2025 02:20:45 GMT
• Title: State Estimation Using Particle Filtering in Adaptive Machine Learning Methods: Integrating Q-Learning and NEAT Algorithms with Noisy Radar Measurements
• Authors: Wonjin Song, Feng Bao,
• Abstract summary: We propose an integrated framework that unifies particle filtering with Q-learning and NEAT to explicitly address the challenge of noisy measurements.<n> Experiments on grid-based navigation and a simulated car environment highlight consistent gains in training stability, final performance, and success rates over baselines lacking advanced filtering.
• Score: 0.8528368686417979
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reliable state estimation is essential for autonomous systems operating in complex, noisy environments. Classical filtering approaches, such as the Kalman filter, can struggle when facing nonlinear dynamics or non-Gaussian noise, and even more flexible particle filters often encounter sample degeneracy or high computational costs in large-scale domains. Meanwhile, adaptive machine learning techniques, including Q-learning and neuroevolutionary algorithms such as NEAT, rely heavily on accurate state feedback to guide learning; when sensor data are imperfect, these methods suffer from degraded convergence and suboptimal performance. In this paper, we propose an integrated framework that unifies particle filtering with Q-learning and NEAT to explicitly address the challenge of noisy measurements. By refining radar-based observations into reliable state estimates, our particle filter drives more stable policy updates (in Q-learning) or controller evolution (in NEAT), allowing both reinforcement learning and neuroevolution to converge faster, achieve higher returns or fitness, and exhibit greater resilience to sensor uncertainty. Experiments on grid-based navigation and a simulated car environment highlight consistent gains in training stability, final performance, and success rates over baselines lacking advanced filtering. Altogether, these findings underscore that accurate state estimation is not merely a preprocessing step, but a vital component capable of substantially enhancing adaptive machine learning in real-world applications plagued by sensor noise.

Related papers

• Efficient Transformed Gaussian Process State-Space Models for Non-Stationary High-Dimensional Dynamical Systems [49.819436680336786]
  Gaussian process state-space models (GPSSMs) have emerged as a powerful framework for modeling dynamical systems.<n>We propose an efficient transformed Gaussian process state-space model (ETGPSSM) to address these limitations.<n>Our approach leverages a single shared Gaussian process (GP) combined with normalizing flows and Bayesian neural networks, enabling efficient modeling of complex, high-dimensional state transitions.
  arXiv  Detail & Related papers  (2025-03-24T03:19:45Z)
• 3D Multi-Object Tracking with Semi-Supervised GRU-Kalman Filter [6.13623925528906]
  3D Multi-Object Tracking (MOT) is essential for intelligent systems like autonomous driving and robotic sensing. We propose a GRU-based MOT method, which introduces a learnable Kalman filter into the motion module. This approach is able to learn object motion characteristics through data-driven learning, thereby avoiding the need for manual model design and model error.
  arXiv  Detail & Related papers  (2024-11-13T08:34:07Z)
• Model-aware reinforcement learning for high-performance Bayesian experimental design in quantum metrology [0.4999814847776097]
  Quantum sensors offer control flexibility during estimation by allowing manipulation by the experimenter across various parameters.<n>We introduce a versatile procedure capable of optimizing a wide range of problems in quantum metrology, estimation, and hypothesis testing.<n>We combine model-aware reinforcement learning (RL) with Bayesian estimation based on particle filtering.
  arXiv  Detail & Related papers  (2023-12-28T12:04:15Z)
• An Ensemble Score Filter for Tracking High-Dimensional Nonlinear Dynamical Systems [10.997994515823798]
  We propose an ensemble score filter (EnSF) for solving high-dimensional nonlinear filtering problems. Unlike existing diffusion models that train neural networks to approximate the score function, we develop a training-free score estimation. EnSF provides surprising performance, compared with the state-of-the-art Local Ensemble Transform Kalman Filter method.
  arXiv  Detail & Related papers  (2023-09-02T16:48:02Z)
• Learning IMM Filter Parameters from Measurements using Gradient Descent [45.335821132209766]
  In intrinsic parameters of targets under track can be completely unobservable until the system is deployed. With state-of-the-art sensor systems growing more and more complex, the number of parameters naturally increases. In this paper, the parameters of an interacting multiple model (IMM) filter are optimized solely using measurements.
  arXiv  Detail & Related papers  (2023-07-13T08:35:40Z)
• Neural optimal feedback control with local learning rules [67.5926699124528]
  A major problem in motor control is understanding how the brain plans and executes proper movements in the face of delayed and noisy stimuli. We introduce a novel online algorithm which combines adaptive Kalman filtering with a model free control approach.
  arXiv  Detail & Related papers  (2021-11-12T20:02:00Z)
• KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics [84.18625250574853]
  We present KalmanNet, a real-time state estimator that learns from data to carry out Kalman filtering under non-linear dynamics. We numerically demonstrate that KalmanNet overcomes nonlinearities and model mismatch, outperforming classic filtering methods.
  arXiv  Detail & Related papers  (2021-07-21T12:26:46Z)
• DANAE: a denoising autoencoder for underwater attitude estimation [1.0312968200748116]
  DANAE is a deep Denoising AutoeNcoder for Attitude Estimation which works on Kalman filter IMU/AHRS data integration. This paper shows that DANAE is robust and reliable, significantly improving the Kalman filter results.
  arXiv  Detail & Related papers  (2020-11-13T10:53:01Z)
• End-To-End Semi-supervised Learning for Differentiable Particle Filters [15.187145925738553]
  Recent advances in incorporating neural networks into particle filters provide the desired flexibility to apply particle filters in real-world applications. Past efforts in optimising such models often require the knowledge of true states which can be expensive to obtain or even unavailable in practice. We present an end-to-end learning objective based upon the maximisation of a pseudo-likelihood function which can improve the estimation of states when large portion of true states are unknown.
  arXiv  Detail & Related papers  (2020-11-11T13:10:11Z)
• Robust Reinforcement Learning with Wasserstein Constraint [49.86490922809473]
  We show the existence of optimal robust policies, provide a sensitivity analysis for the perturbations, and then design a novel robust learning algorithm. The effectiveness of the proposed algorithm is verified in the Cart-Pole environment.
  arXiv  Detail & Related papers  (2020-06-01T13:48:59Z)
• Neural Networks Versus Conventional Filters for Inertial-Sensor-based Attitude Estimation [1.0957528713294873]
  Inertial measurement units are commonly used to estimate the attitude of moving objects. nonlinear filter approaches have been proposed for solving the inherent sensor fusion problem. We investigate what extent these limitations can be overcome by means of artificial neural networks.
  arXiv  Detail & Related papers  (2020-05-14T11:59:19Z)
• Tracking Performance of Online Stochastic Learners [57.14673504239551]
  Online algorithms are popular in large-scale learning settings due to their ability to compute updates on the fly, without the need to store and process data in large batches. When a constant step-size is used, these algorithms also have the ability to adapt to drifts in problem parameters, such as data or model properties, and track the optimal solution with reasonable accuracy. We establish a link between steady-state performance derived under stationarity assumptions and the tracking performance of online learners under random walk models.
  arXiv  Detail & Related papers  (2020-04-04T14:16:27Z)
• Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems [91.43582419264763]
  We study the problem of system identification and adaptive control in partially observable linear dynamical systems. We present the first model estimation method with finite-time guarantees in both open and closed-loop system identification. We show that AdaptOn is the first algorithm that achieves $textpolylogleft(Tright)$ regret in adaptive control of unknown partially observable linear dynamical systems.
  arXiv  Detail & Related papers  (2020-03-25T06:00:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.