Multimodal-to-Text Prompt Engineering in Large Language Models Using Feature Embeddings for GNSS Interference Characterization

• URL: http://arxiv.org/abs/2501.05079v2
• Date: Wed, 15 Jan 2025 20:46:44 GMT
• Title: Multimodal-to-Text Prompt Engineering in Large Language Models Using Feature Embeddings for GNSS Interference Characterization
• Authors: Harshith Manjunath, Lucas Heublein, Tobias Feigl, Felix Ott,
• Abstract summary: Large language models (LLMs) are advanced AI systems applied across various domains, including NLP, information retrieval, and recommendation systems. interference monitoring is essential to ensure the reliability of vehicle localization on roads. Our pipeline outperforms state-of-the-art machine learning models in interference classification tasks.
• Score: 2.469551405169408
• License:
• Abstract: Large language models (LLMs) are advanced AI systems applied across various domains, including NLP, information retrieval, and recommendation systems. Despite their adaptability and efficiency, LLMs have not been extensively explored for signal processing tasks, particularly in the domain of global navigation satellite system (GNSS) interference monitoring. GNSS interference monitoring is essential to ensure the reliability of vehicle localization on roads, a critical requirement for numerous applications. However, GNSS-based positioning is vulnerable to interference from jamming devices, which can compromise its accuracy. The primary objective is to identify, classify, and mitigate these interferences. Interpreting GNSS snapshots and the associated interferences presents significant challenges due to the inherent complexity, including multipath effects, diverse interference types, varying sensor characteristics, and satellite constellations. In this paper, we extract features from a large GNSS dataset and employ LLaVA to retrieve relevant information from an extensive knowledge base. We employ prompt engineering to interpret the interferences and environmental factors, and utilize t-SNE to analyze the feature embeddings. Our findings demonstrate that the proposed method is capable of visual and logical reasoning within the GNSS context. Furthermore, our pipeline outperforms state-of-the-art machine learning models in interference classification tasks.

Related papers

• Reliable Explainability of Deep Learning Spatial-Spectral Classifiers for Improved Semantic Segmentation in Autonomous Driving [1.474723404975345]
  Hyperspectral imagery (HSI) and deep neural networks (DNNs) can strengthen the accuracy of intelligent vision systems. To advance research in such safety-critical systems, determining the precise contribution of spectral information to complex DNNs' output is needed. We propose an alternative approach by leveraging the data provided by activations and weights from relevant DNN layers to better capture the relationship between input features and predictions.
  arXiv  Detail & Related papers  (2025-02-20T10:11:27Z)
• Evaluating ML Robustness in GNSS Interference Classification, Characterization & Localization [42.14439854721613]
  Jamming devices disrupt signals from the global navigation satellite system (GNSS) This paper introduces an extensive dataset comprising snapshots obtained from a low-frequency antenna. Our objective is to assess the resilience of machine learning (ML) models against environmental changes.
  arXiv  Detail & Related papers  (2024-09-23T15:20:33Z)
• Renormalized Connection for Scale-preferred Object Detection in Satellite Imagery [51.83786195178233]
  We design a Knowledge Discovery Network (KDN) to implement the renormalization group theory in terms of efficient feature extraction. Renormalized connection (RC) on the KDN enables synergistic focusing'' of multi-scale features. RCs extend the multi-level feature's divide-and-conquer'' mechanism of the FPN-based detectors to a wide range of scale-preferred tasks.
  arXiv  Detail & Related papers  (2024-09-09T13:56:22Z)
• A Survey of Machine Learning Techniques for Improving Global Navigation Satellite Systems [0.0]
  This paper highlights recent advances in Machine Learning (ML) and its potential to address these limitations. It covers a broad range of ML methods, including supervised learning, unsupervised learning, deep learning, and hybrid approaches. The survey provides insights into positioning applications related to such as signal analysis, anomaly detection, multi-sensor integration, prediction, and accuracy enhancement using ML.
  arXiv  Detail & Related papers  (2024-03-29T18:31:50Z)
• Extending RAIM with a Gaussian Mixture of Opportunistic Information [1.9688858888666714]
  Original receiver autonomous integrity monitoring (RAIM) was not designed for securing. We extend RAIM by incorporating all opportunistic information, i.e., measurements from terrestrial infrastructures and onboard sensors. The objective is to assess the likelihood of spoofing by analyzing locations derived from extended RAIM solutions.
  arXiv  Detail & Related papers  (2024-02-05T19:03:18Z)
• Learning-based NLOS Detection and Uncertainty Prediction of GNSS Observations with Transformer-Enhanced LSTM Network [2.798138034569478]
  This work proposes a deeplearning-based method to detect NLOS and predict errors by analyzing pseudo-temporal modeling problem. We use datasets from Hong Kong and Aachen to train and evaluate the proposed network. We show that the proposed method avoids trajectory divergence in real-world vehicle localization by classifying and excluding NLOS observations.
  arXiv  Detail & Related papers  (2023-09-01T14:17:02Z)
• Efficient Real-time Smoke Filtration with 3D LiDAR for Search and Rescue with Autonomous Heterogeneous Robotic Systems [56.838297900091426]
  Smoke and dust affect the performance of any mobile robotic platform due to their reliance on onboard perception systems. This paper proposes a novel modular computation filtration pipeline based on intensity and spatial information.
  arXiv  Detail & Related papers  (2023-08-14T16:48:57Z)
• Signal Detection in MIMO Systems with Hardware Imperfections: Message Passing on Neural Networks [101.59367762974371]
  In this paper, we investigate signal detection in multiple-input-multiple-output (MIMO) communication systems with hardware impairments. It is difficult to train a deep neural network (DNN) with limited pilot signals, hindering its practical applications. We design an efficient message passing based Bayesian signal detector, leveraging the unitary approximate message passing (UAMP) algorithm.
  arXiv  Detail & Related papers  (2022-10-08T04:32:58Z)
• RRNet: Relational Reasoning Network with Parallel Multi-scale Attention for Salient Object Detection in Optical Remote Sensing Images [82.1679766706423]
  Salient object detection (SOD) for optical remote sensing images (RSIs) aims at locating and extracting visually distinctive objects/regions from the optical RSIs. We propose a relational reasoning network with parallel multi-scale attention for SOD in optical RSIs. Our proposed RRNet outperforms the existing state-of-the-art SOD competitors both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2021-10-27T07:18:32Z)
• On the benefits of robust models in modulation recognition [53.391095789289736]
  Deep Neural Networks (DNNs) using convolutional layers are state-of-the-art in many tasks in communications. In other domains, like image classification, DNNs have been shown to be vulnerable to adversarial perturbations. We propose a novel framework to test the robustness of current state-of-the-art models.
  arXiv  Detail & Related papers  (2021-03-27T19:58:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.