Adaptive Kalman-Informed Transformer

• URL: http://arxiv.org/abs/2401.09987v2
• Date: Fri, 07 Mar 2025 19:48:55 GMT
• Title: Adaptive Kalman-Informed Transformer
• Authors: Nadav Cohen, Itzik Klein,
• Abstract summary: The extended Kalman filter (EKF) is a widely adopted method for sensor fusion in navigation applications.<n>We derive an adaptive Kalman-informed transformer (A-KIT) designed to learn the varying process noise covariance online.<n>We show that A-KIT outperforms the conventional EKF by more than 49.5% and model-based adaptive EKF by an average of 35.4% in terms of position accuracy.
• Score: 13.221163846643607
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The extended Kalman filter (EKF) is a widely adopted method for sensor fusion in navigation applications. A crucial aspect of the EKF is the online determination of the process noise covariance matrix reflecting the model uncertainty. While common EKF implementation assumes a constant process noise, in real-world scenarios, the process noise varies, leading to inaccuracies in the estimated state and potentially causing the filter to diverge. Model-based adaptive EKF methods were proposed and demonstrated performance improvements to cope with such situations, highlighting the need for a robust adaptive approach. In this paper, we derive an adaptive Kalman-informed transformer (A-KIT) designed to learn the varying process noise covariance online. Built upon the foundations of the EKF, A-KIT utilizes the well-known capabilities of set transformers, including inherent noise reduction and the ability to capture nonlinear behavior in the data. This approach is suitable for any application involving the EKF. In a case study, we demonstrate the effectiveness of A-KIT in nonlinear fusion between a Doppler velocity log and inertial sensors. This is accomplished using real data recorded from sensors mounted on an autonomous underwater vehicle operating in the Mediterranean Sea. We show that A-KIT outperforms the conventional EKF by more than 49.5% and model-based adaptive EKF by an average of 35.4% in terms of position accuracy.

Related papers

• Lightweight Task-Oriented Semantic Communication Empowered by Large-Scale AI Models [66.57755931421285]
  Large-scale artificial intelligence (LAI) models pose significant challenges for real-time communication scenarios.<n>This paper proposes utilizing knowledge distillation (KD) techniques to extract and condense knowledge from LAI models.<n>We propose a fast distillation method featuring a pre-stored compression mechanism that eliminates the need for repetitive inference.
  arXiv  Detail & Related papers  (2025-06-16T08:42:16Z)
• Adaptive folding and noise filtering for robust quantum error mitigation [0.0]
  This paper presents noise-adaptive folding, a technique that enhances zero-noise extrapolation (ZNE)<n>We introduce two filtering methods: one relies on measuring error strength, while the other utilizes statistical filtering to improve the extrapolation process.<n>Our findings demonstrate that these adaptive methods effectively strengthen error mitigation against noise fluctuations, thereby enhancing the precision and reliability of quantum computations.
  arXiv  Detail & Related papers  (2025-05-07T14:35:01Z)
• EnviKal-Loc: Sub-10m Indoor LoRaWAN Localization using an Environmental-Aware Path Loss and Adaptive RSSI Smoothing [6.8093214146903875]
  This paper proposes a lightweight but robust approach to achieve sub-10 m accuracy in LoRaWAN localization.<n>Our methodology augments conventional models with critical LoRaWAN parameters.<n>An adaptive Kalman filter reduces RSSI fluctuations, isolating persistent trends from momentary noise.
  arXiv  Detail & Related papers  (2025-05-02T11:00:40Z)
• Theoretical Insights in Model Inversion Robustness and Conditional Entropy Maximization for Collaborative Inference Systems [89.35169042718739]
  collaborative inference enables end users to leverage powerful deep learning models without exposure of sensitive raw data to cloud servers. Recent studies have revealed that these intermediate features may not sufficiently preserve privacy, as information can be leaked and raw data can be reconstructed via model inversion attacks (MIAs) This work first theoretically proves that the conditional entropy of inputs given intermediate features provides a guaranteed lower bound on the reconstruction mean square error (MSE) under any MIA. Then, we derive a differentiable and solvable measure for bounding this conditional entropy based on the Gaussian mixture estimation and propose a conditional entropy algorithm to enhance the inversion robustness
  arXiv  Detail & Related papers  (2025-03-01T07:15:21Z)
• Noise-Adaptive Conformal Classification with Marginal Coverage [53.74125453366155]
  We introduce an adaptive conformal inference method capable of efficiently handling deviations from exchangeability caused by random label noise. We validate our method through extensive numerical experiments demonstrating its effectiveness on synthetic and real data sets.
  arXiv  Detail & Related papers  (2025-01-29T23:55:23Z)
• Adaptive Conformal Inference by Betting [51.272991377903274]
  We consider the problem of adaptive conformal inference without any assumptions about the data generating process. Existing approaches for adaptive conformal inference are based on optimizing the pinball loss using variants of online gradient descent. We propose a different approach for adaptive conformal inference that leverages parameter-free online convex optimization techniques.
  arXiv  Detail & Related papers  (2024-12-26T18:42:08Z)
• Test-Time Model Adaptation with Only Forward Passes [68.11784295706995]
  Test-time adaptation has proven effective in adapting a given trained model to unseen test samples with potential distribution shifts. We propose a test-time Forward-Optimization Adaptation (FOA) method. FOA runs on quantized 8-bit ViT, outperforms gradient-based TENT on full-precision 32-bit ViT, and achieves an up to 24-fold memory reduction on ImageNet-C.
  arXiv  Detail & Related papers  (2024-04-02T05:34:33Z)
• Thwarting Cybersecurity Attacks with Explainable Concept Drift [10.517955982799553]
  Cyber-security attacks pose a significant threat to the operation of autonomous systems. This paper proposes a Feature Drift Explanation (FDE) module to identify the drifting features. FDE successfully identifies 85.77 % of drifting features and showcases its utility in the DL adaptation method.
  arXiv  Detail & Related papers  (2024-03-18T20:20:00Z)
• Joint State Estimation and Noise Identification Based on Variational Optimization [8.536356569523127]
  A novel adaptive Kalman filter method based on conjugate-computation variational inference, referred to as CVIAKF, is proposed. The effectiveness of CVIAKF is validated through synthetic and real-world datasets of maneuvering target tracking.
  arXiv  Detail & Related papers  (2023-12-15T07:47:03Z)
• Ensemble Kalman Filtering Meets Gaussian Process SSM for Non-Mean-Field and Online Inference [47.460898983429374]
  We introduce an ensemble Kalman filter (EnKF) into the non-mean-field (NMF) variational inference framework to approximate the posterior distribution of the latent states. This novel marriage between EnKF and GPSSM not only eliminates the need for extensive parameterization in learning variational distributions, but also enables an interpretable, closed-form approximation of the evidence lower bound (ELBO) We demonstrate that the resulting EnKF-aided online algorithm embodies a principled objective function by ensuring data-fitting accuracy while incorporating model regularizations to mitigate overfitting.
  arXiv  Detail & Related papers  (2023-12-10T15:22:30Z)
• Uncertainty Guided Adaptive Warping for Robust and Efficient Stereo Matching [77.133400999703]
  Correlation based stereo matching has achieved outstanding performance. Current methods with a fixed model do not work uniformly well across various datasets. This paper proposes a new perspective to dynamically calculate correlation for robust stereo matching.
  arXiv  Detail & Related papers  (2023-07-26T09:47:37Z)
• Federated Conformal Predictors for Distributed Uncertainty Quantification [83.50609351513886]
  Conformal prediction is emerging as a popular paradigm for providing rigorous uncertainty quantification in machine learning. In this paper, we extend conformal prediction to the federated learning setting. We propose a weaker notion of partial exchangeability, better suited to the FL setting, and use it to develop the Federated Conformal Prediction framework.
  arXiv  Detail & Related papers  (2023-05-27T19:57:27Z)
• Outlier-Insensitive Kalman Filtering Using NUV Priors [24.413595920205907]
  In practice, observations are corrupted by outliers, severely impairing the Kalman filter (KF)s performance. In this work, an outlier-insensitive KF is proposed, where is achieved by modeling each potential outlier as a normally distributed random variable with unknown variance (NUV) The NUVs variances are estimated online, using both expectation-maximization (EM) and alternating robustness (AM)
  arXiv  Detail & Related papers  (2022-10-12T11:00:13Z)
• Data-Driven Chance Constrained AC-OPF using Hybrid Sparse Gaussian Processes [57.70237375696411]
  The paper proposes a fast data-driven setup that uses the sparse and hybrid Gaussian processes (GP) framework to model the power flow equations with input uncertainty. We advocate the efficiency of the proposed approach by a numerical study over multiple IEEE test cases showing up to two times faster and more accurate solutions.
  arXiv  Detail & Related papers  (2022-08-30T09:27:59Z)
• A Hybrid Model and Learning-Based Adaptive Navigation Filter [0.0]
  We propose a hybrid model and learning-based adaptive navigation filter. We show that the proposed method obtained an improvement of 25% in the position error.
  arXiv  Detail & Related papers  (2022-06-14T17:10:47Z)
• Variational Kalman Filtering with Hinf-Based Correction for Robust Bayesian Learning in High Dimensions [2.294014185517203]
  We address the problem of convergence of sequential variational inference filter (VIF) through the application of a robust variational objective and Hinf-norm based correction. A novel VIF- Hinf recursion that employs consecutive variational inference and Hinf based optimization steps is proposed.
  arXiv  Detail & Related papers  (2022-04-27T17:38:13Z)
• Adaptive variational preparation of the Fermi-Hubbard eigenstates [0.0]
  We prepare highly accurate ground states of the Fermi-Hubbard model for small grids up to 6 sites (12bits) We show this adaptive method outperforms the non-adaptive counterpart in terms of fewer variational parameters, short gate depth, and scaling with the system size. We also demonstrate the application of adaptive variational methods by preparing excited states and Green functions using a proposed ADAPT-SSVQE algorithm.
  arXiv  Detail & Related papers  (2021-09-24T18:00:05Z)
• Layer Pruning on Demand with Intermediate CTC [50.509073206630994]
  We present a training and pruning method for ASR based on the connectionist temporal classification (CTC) We show that a Transformer-CTC model can be pruned in various depth on demand, improving real-time factor from 0.005 to 0.002 on GPU.
  arXiv  Detail & Related papers  (2021-06-17T02:40:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.