Lost in the Vibrations: Vision Language Models Fail the Dynamic Gauges Test

• URL: http://arxiv.org/abs/2604.22829v1
• Date: Sun, 19 Apr 2026 15:45:13 GMT
• Title: Lost in the Vibrations: Vision Language Models Fail the Dynamic Gauges Test
• Abstract summary: Vision-Language Models (VLMs) have demonstrated potential in zero-shot instrument recognition.<n>Their deployment in measurement systems remains constrained by an inherent inability to accurately analyze high-frequency temporal events and needle vibrations.<n>This paper evaluates state-of-the-art models, including GPT-5 and Gemini 3, against the strict requirements of metrology and uncertainty quantification.
• Score: 4.017289687696229
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The digital transformation of industrial manufacturing increasingly relies on the ability of autonomous robots to interact with legacy infrastructure, particularly analog gauges. While Vision-Language Models (VLMs) have demonstrated potential in zero-shot instrument recognition, their deployment in measurement systems remains constrained by an inherent inability to accurately analyze high-frequency temporal events and needle vibrations. This paper evaluates state-of-the-art models, including GPT-5 and Gemini 3, against the strict requirements of metrology and uncertainty quantification. To facilitate this evaluation, we introduce a novel dataset comprising video sequences of various gauge types: circular, linear, and Vernier, under diverse motion speed profiles. Our findings indicate that current VLMs exhibit limited ability in interpreting needle trajectories and scale semantics, failing to provide the traceability and reliability needed for safety-critical monitoring. The results demonstrate that these models have not yet achieved the performance necessary to be classified as trustworthy synthetic instruments under existing IEEE and ISO standards.

Related papers

• Vision-Language-Action Models: Experimental Insights from a Real-World UR5 Platform [0.0]
  This project investigates whether recent Vision-Language-Action (VLA) models can be transferred from controlled research benchmarks to a real-world robotic platform.<n>The work integrates real-robot data acquisition, dataset engineering, and the fine-tuning and deployment of OpenVLA and OpenVLA-OFT models.
  arXiv  Detail & Related papers  (2026-06-29T15:23:34Z)
• U-STS-LLM A Unified Spatio-Temporal Steered Large Language Model for Traffic Prediction and Imputation [6.194596351746244]
  We present U-STS-LLM, a unified framework built on a-temporal-ly steered LLM.<n>Our core innovation is a Dynamic S-Lo Concurrent Attention Bias Generator that synthesizes a persistent functional graph with transient nodal states.<n>Experiments on real-world cellular datasets demonstrate that U-STS-LLM establishes new state-of-the-art performance in both long-horizon forecasting and high-missing-rate imputation.
  arXiv  Detail & Related papers  (2026-05-12T08:16:22Z)
• Causal Parametric Drift Simulation: A Digital Twin Framework for Classifier Robustness Evaluation [0.0]
  Machine learning classifiers in dynamic environments face concept drift.<n>We propose a framework that complements existing drift detection by leveraging Structural Causal Models as "Digital Twins"<n>Our technique, Causal Parametric Drift Simulation, stress-tests classifiers to identify vulnerabilities before deployment.
  arXiv  Detail & Related papers  (2026-05-10T17:28:49Z)
• Navigating by Old Maps: The Pitfalls of Static Mechanistic Localization in LLM Post-Training [59.49057547517285]
  We systematically track the structural evolution of Transformer circuits throughout the supervised fine-tuning process.<n>Our empirical results reveal that circuits inherently exhibit "Free Evolution" during parameter updates.<n>This work underscores the necessity of "foresight" in mechanistic localization and proposes a framework for future research.
  arXiv  Detail & Related papers  (2026-05-07T11:59:52Z)
• Same Answer, Different Representations: Hidden instability in VLMs [65.36933543377346]
  We introduce a representation-aware and frequency-aware evaluation framework that measures internal embedding drift, spectral sensitivity, and structural smoothness.<n>We apply this framework to modern Vision Language Models (VLMs) across the SEEDBench, MMMU, and POPE datasets.
  arXiv  Detail & Related papers  (2026-02-06T12:24:26Z)
• Reshaping Action Error Distributions for Reliable Vision-Language-Action Models [69.38615670891038]
  In robotic manipulation, vision-language-action (VLA) models have emerged as a promising paradigm for learning generalizable and scalable robot policies.<n>We focus on continuous-action VLA models and move beyond conventional MSE-based regression by reshaping action error distributions during training.<n>We evaluate our approaches across standard, few-shot, and noisy settings on multiple representative VLA architectures.
  arXiv  Detail & Related papers  (2026-02-04T05:37:09Z)
• Simplicity Prevails: The Emergence of Generalizable AIGI Detection in Visual Foundation Models [15.709482146201283]
  A simple linear classifier trained on the frozen features of modern Vision Foundation Models establishes a new state-of-the-art.<n>We show that this baseline matches specialized detectors on standard benchmarks but also decisively outperforms them on in-the-wild datasets.<n>We conclude by advocating for a paradigm shift in AI forensics, moving from overfitting on static benchmarks to harnessing the evolving world knowledge of foundation models for real-world reliability.
  arXiv  Detail & Related papers  (2026-02-02T07:20:02Z)
• LaST$_{0}$: Latent Spatio-Temporal Chain-of-Thought for Robotic Vision-Language-Action Model [102.60980325911106]
  We propose a framework that enables efficient reasoning before acting through a Latent Spatio-Temporal Chain-of-Thought (CoT)<n>Specifically, we introduce a token-efficient latent CoT space that models future visual dynamics, 3D structural information, and robot proprioceptive states, and further extends these representations across time to enable temporally consistent implicit reasoning trajectories.
  arXiv  Detail & Related papers  (2026-01-08T18:59:53Z)
• Explainability of Point Cloud Neural Networks Using SMILE: Statistical Model-Agnostic Interpretability with Local Explanations [0.0]
  This study explores the implementation of SMILE, a novel explainability method originally designed for deep neural networks, on point cloud-based models. The approach demonstrates superior performance in terms of fidelity loss, R2 scores, and robustness across various kernel widths, perturbation numbers, and clustering configurations. The study further identifies dataset biases in the classification of the 'person' category, emphasizing the necessity for more comprehensive datasets in safety-critical applications.
  arXiv  Detail & Related papers  (2024-10-20T12:13:59Z)
• Benchmarks as Microscopes: A Call for Model Metrology [76.64402390208576]
  Modern language models (LMs) pose a new challenge in capability assessment. To be confident in our metrics, we need a new discipline of model metrology.
  arXiv  Detail & Related papers  (2024-07-22T17:52:12Z)
• Real-Time Anomaly Detection and Reactive Planning with Large Language Models [18.57162998677491]
  Foundation models, e.g., large language models (LLMs), trained on internet-scale data possess zero-shot capabilities. We present a two-stage reasoning framework that incorporates the judgement regarding potential anomalies into a safe control framework. This enables our monitor to improve the trustworthiness of dynamic robotic systems, such as quadrotors or autonomous vehicles.
  arXiv  Detail & Related papers  (2024-07-11T17:59:22Z)
• Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigation [67.18144414660681]
  We propose a Fast-Slow Test-Time Adaptation (FSTTA) approach for online Vision-and-Language Navigation (VLN) Our method obtains impressive performance gains on four popular benchmarks.
  arXiv  Detail & Related papers  (2023-11-22T07:47:39Z)
• Model Monitoring and Robustness of In-Use Machine Learning Models: Quantifying Data Distribution Shifts Using Population Stability Index [2.578242050187029]
  We focus on a computer vision example related to autonomous driving and aim at detecting shifts that occur as a result of adding noise to images. We use the population stability index (PSI) as a measure of presence and intensity of shift and present results of our empirical experiments.
  arXiv  Detail & Related papers  (2023-02-01T22:06:31Z)
• SyMetric: Measuring the Quality of Learnt Hamiltonian Dynamics Inferred from Vision [73.26414295633846]
  A recently proposed class of models attempts to learn latent dynamics from high-dimensional observations. Existing methods rely on image reconstruction quality, which does not always reflect the quality of the learnt latent dynamics. We develop a set of new measures, including a binary indicator of whether the underlying Hamiltonian dynamics have been faithfully captured.
  arXiv  Detail & Related papers  (2021-11-10T23:26:58Z)
• Deep Switching State Space Model (DS$^3$M) for Nonlinear Time Series Forecasting with Regime Switching [2.8579459256051316]
  We introduce a novel modeling framework known as the Deep Switching State Space Model (DS$3$M)<n>This framework is engineered to make accurate forecasts for such time series while adeptly identifying the irregular regimes hidden within the dynamics.<n>We validate the effectiveness and regime identification capabilities of DS$3$M through short- and long-term forecasting tests on a wide array of simulated and real-world datasets.
  arXiv  Detail & Related papers  (2021-06-04T08:25:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.