FabriMAE I Trust Myself? Self-Evaluating VLA Action Generation with Markov Attention Entropy

• URL: http://arxiv.org/abs/2608.16697v1
• Date: Mon, 17 Aug 2026 15:14:53 GMT
• Title: FabriMAE I Trust Myself? Self-Evaluating VLA Action Generation with Markov Attention Entropy
• Abstract summary: Vision-Language-Action models (VLAs) integrate visual perception, language instruction, and action generation into end-to-end policies.<n>Existing methods either rely on expert annotations or estimate uncertainty only from output statistics.<n>We propose MAE (Markov Attention Entropy), a self-evaluation framework that converts internal attention signals into architecture-aware reliability scores.
• Score: 63.26759696630558
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Vision-Language-Action models (VLAs) integrate visual perception, language instruction, and action generation into end-to-end policies across heterogeneous architectures. However, enabling VLAs to self-evaluate their action generation reliability without external supervision remains a major challenge. Existing methods either rely on expert annotations or estimate uncertainty only from output statistics, largely ignoring internal signals. In this work, we observe that internal visual modality entropy exhibits consistent distinctions between successful and failed tasks across heterogeneous VLAs. Although VLAs' architectures differ in their action generation, we show that they share a common latent action generation abstraction evolving under visual perception, language instruction, and state input, which we formulate as a Conditional Generative Markov Chain. Based on this formulation, we propose MAE (Markov Attention Entropy), a self-evaluation framework that directly converts internal attention signals into architecture-aware reliability scores, and introduce LIBERO-Reflect, a 4,000-episode benchmark combining 2,000 standard episodes and 2,000 challenging episodes across four subsets. Extensive experiments across heterogeneous VLA architectures and diverse scenarios show that MAE consistently outperforms state-of-the-art baselines on AUPR, AUROC, and FPR@95. We further instantiate FabriMAE for verifier-free test-time action selection, showing that MAE-guided multiple sampling improves PI-family robustness on LIBERO-Plus with small observed runtime overhead.

Related papers

• Towards Generalizable Visually Grounded Exploration of Household Devices [74.66117923766632]
  Key bottleneck of realizing general autonomous embodied agents lies in Generalizable Visually Grounded Exploration.<n>We introduce VGEBench, a benchmark designed to evaluate the generalizable visually grounded exploration capabilities of Vision-Language Models.<n>Unlike static datasets, we construct a Logic-Driven State Machine framework. This framework simulates multi-turn interaction loops, compelling agents to achieve goals by active visual perception and feedback-driven correction.
  arXiv  Detail & Related papers  (2026-09-01T07:42:45Z)
• Unified Hallucination Fuzzing for Multimodal Large Language Models [44.4785399881315]
  Hallucination remains a persistent challenge for Multimodal Large Language Models (MLLMs)<n>Existing evaluations, predominantly based on static benchmarks, suffer from narrow taxonomical coverage and rapid performance saturation.<n>We present a systematic evaluation framework integrating a comprehensive benchmark with self-evolving stress testing.
  arXiv  Detail & Related papers  (2026-07-15T17:57:37Z)
• AlphaGRPO: Unlocking Self-Reflective Multimodal Generation in UMMs via Decompositional Verifiable Reward [52.7128440950672]
  We propose AlphaGRPO to enhance multimodal generation capabilities without an additional cold-start stage.<n>Our approach unlocks the model's intrinsic potential to perform advanced reasoning tasks.<n>To address the challenge of providing stable supervision for real-world multimodal generation, we introduce the Decompositional Verifiable Reward (DVReward)
  arXiv  Detail & Related papers  (2026-05-12T17:59:47Z)
• SMFormer: Empowering Self-supervised Stereo Matching via Foundation Models and Data Augmentation [55.67206878777881]
  We propose SMFormer, a framework integrating more reliable self-supervision guided by the Vision Foundation Model (VFM) and data augmentation.<n> SMFormer achieves state-of-the-art (SOTA) performance among self-supervised methods and even competes on par with supervised ones.
  arXiv  Detail & Related papers  (2026-04-11T13:56:41Z)
• Observing and Controlling Features in Vision-Language-Action Models [20.193036403223903]
  Vision-Language-Action Models (VLAs) have shown remarkable progress towards embodied intelligence.<n>We introduce and analyze two main concepts: feature-observability and feature-controllability.<n>Our results show that targeted, lightweight interventions can reliably steer a robot's behavior.
  arXiv  Detail & Related papers  (2026-03-05T18:53:50Z)
• VAUQ: Vision-Aware Uncertainty Quantification for LVLM Self-Evaluation [22.921677603408188]
  Large Vision-Language Models (LVLMs) frequently hallucinate, limiting their safe deployment in real-world applications.<n>We propose VAUQ, a vision-aware uncertainty quantification framework for LVLM self-evaluation.<n>VAUQ explicitly measures how strongly a model's output depends on visual evidence.
  arXiv  Detail & Related papers  (2026-02-24T16:11:14Z)
• Reshaping Action Error Distributions for Reliable Vision-Language-Action Models [69.38615670891038]
  In robotic manipulation, vision-language-action (VLA) models have emerged as a promising paradigm for learning generalizable and scalable robot policies.<n>We focus on continuous-action VLA models and move beyond conventional MSE-based regression by reshaping action error distributions during training.<n>We evaluate our approaches across standard, few-shot, and noisy settings on multiple representative VLA architectures.
  arXiv  Detail & Related papers  (2026-02-04T05:37:09Z)
• ReViP: Reducing False Completion in Vision-Language-Action Models with Vision-Proprioception Rebalance [50.05984919728878]
  We present ReViP, a novel VLA framework with Vision-Proprioception Rebalance to enhance visual grounding and robustness under perturbations.<n>Specifically, we use an external VLM as a task-stage observer to extract real-time task-centric visual cues from visual observations.<n>To evaluate false completion, we propose the first False-Completion Benchmark Suite built on LIBERO with controlled settings such as Object-Drop.
  arXiv  Detail & Related papers  (2026-01-23T11:31:07Z)
• HARMONY: Hidden Activation Representations and Model Output-Aware Uncertainty Estimation for Vision-Language Models [42.91752946934796]
  Uncertainty Estimation plays a central role in quantifying the reliability of model outputs.<n>Most existing probability-based UE approaches rely on output probability distributions aggregating token probabilities into a single uncertainty score.<n>We propose a novel UE framework, HARMONY, that jointly leverages fused multimodal information in model activations and the output distribution of the VLM.
  arXiv  Detail & Related papers  (2025-10-25T05:45:18Z)
• Self-Consistency as a Free Lunch: Reducing Hallucinations in Vision-Language Models via Self-Reflection [71.8243083897721]
  Vision-language models often hallucinate details, generating non-existent objects or inaccurate attributes that compromise output reliability.<n>We present a novel framework that leverages the model's self-consistency between long responses and short answers to generate preference pairs for training.
  arXiv  Detail & Related papers  (2025-09-27T10:37:11Z)
• SUDER: Self-Improving Unified Large Multimodal Models for Understanding and Generation with Dual Self-Rewards [55.99492656542475]
  We propose textbfSUDER (textbfSelf-improving textbfUnified LMMs with textbfDual stextbfElf-textbfRewards), a framework reinforcing the understanding and generation capabilities of LMMs.
  arXiv  Detail & Related papers  (2025-06-09T17:38:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.