Trustworthy Visual Predicates for Robust Manipulation Understanding under Degradation

• URL: http://arxiv.org/abs/2606.08121v1
• Date: Sat, 06 Jun 2026 11:53:42 GMT
• Title: Trustworthy Visual Predicates for Robust Manipulation Understanding under Degradation
• Abstract summary: This paper introduces a predicate-level reliability framework for robust manipulation understanding under blur, occlusion, illumination change, low resolution, frame dropping, and detection noise.<n>Experiments on controlled manipulation videos and public egocentric or bimanual datasets, including VISOR/EPIC-KITCHENS, H2O, and ARCTIC, show that predicate failures are structured rather than uniform.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Manipulation understanding requires reliable relational evidence, such as contact, support, containment, motion coupling, grasp, release, and active-hand involvement. Although these visual predicates are widely used in event-chain, graph-based, and neuro-symbolic models, their reliability under visual degradation is rarely analyzed directly. This paper introduces a predicate-level reliability framework for robust manipulation understanding under blur, occlusion, illumination change, low resolution, frame dropping, and detection noise. The framework defines a structured predicate vocabulary, confidence-aware predicate estimation, and reliability metrics for predicate preservation, degradation sensitivity, temporal consistency, confidence-weighted stability, and downstream impact. Experiments on controlled manipulation videos and public egocentric or bimanual datasets, including VISOR/EPIC-KITCHENS, H2O, and ARCTIC, show that predicate failures are structured rather than uniform. Static spatial predicates remain comparatively robust, whereas contact-sensitive, dynamic, and derived predicates such as grasp and release are more fragile. Under severe degradation, detection noise, occlusion, and frame dropping cause the strongest reliability losses. Downstream analysis shows that degraded predicates reduce manipulation-understanding accuracy from 0.89 to 0.58, while removing confidence weighting under moderate degradation reduces accuracy from 0.74 to 0.64. These results show that predicate reliability provides a diagnostic layer between visual perception and structured manipulation reasoning.

Related papers

• Robustifying Vision-Language Models via Test-Time Prompt Adaptation [23.05168102474528]
  We propose RITA, a test-tIme prompt-TAdaptation framework that shifts from sample-level estimates to distribution-level alignment.<n>Specifically, RITA employs optimal transport to align the distribution of augmented visual features with textual prototypes, mitigating adversarial outliers and rectifying cross-modal semantic misalignment.
  arXiv  Detail & Related papers  (2026-07-10T14:19:42Z)
• Task-Aligned Stability Analysis of Vision-Language Models for Autonomous Driving Hazard Detection [0.0]
  We study whether corruption-induced embedding drift predicts changes in a task-aligned hazard score derived from CLIP image-text similarities.<n>Using controlled corruptions on BDD100K road scenes, we compare embedding drift against margin drift.<n>Some families exhibit strong coupling between representation drift and decision drift, while others induce hazardous decision instability despite relatively modest embedding change.
  arXiv  Detail & Related papers  (2026-06-10T10:20:14Z)
• Stabilizing Temporal Inference Dynamics for Online Surgical Phase Recognition [42.76777398447746]
  Online Surgical Phase Recognition (SPR) models can reach high frame-wise accuracy, yet their predictions often lack temporal stability.<n>We show that this instability is not random noise but arises from two mechanisms: early misclassifications corrupt temporal feature states and propagate forward to form error cascades.<n>We propose a unified Train-Inference-Evaluation framework that explicitly stabilizes temporal inference dynamics using model-agnostic, plug-and-play components.
  arXiv  Detail & Related papers  (2026-05-11T16:56:15Z)
• DO-Bench: An Attributable Benchmark for Diagnosing Object Hallucination in Vision-Language Models [17.5535076530203]
  Object level hallucination remains a central reliability challenge for vision language models.<n>We introduce DO-Bench, a controlled diagnostic benchmark that isolates sources through structured interventions.<n>We show that object hallucination reflects heterogeneous, mechanism dependent failure patterns beyond aggregate accuracy.
  arXiv  Detail & Related papers  (2026-04-18T06:54:02Z)
• Towards Trustworthy Depression Estimation via Disentangled Evidential Learning [50.22167852149165]
  We propose EviDep, an evidential learning framework that jointly quantifies depression severity.<n>EviDep enforces strict information integrity to guarantee robust evidence synthesis.<n>It achieves state-of-the-art predictive accuracy and superior uncertainty calibration, delivering a robust fail-safe mechanism for trustworthy clinical screening.
  arXiv  Detail & Related papers  (2026-04-17T13:27:11Z)
• INFACT: A Diagnostic Benchmark for Induced Faithfulness and Factuality Hallucinations in Video-LLMs [69.11382230669491]
  hallucinations are outputs that contradict either video evidence (faithfulness) or verifiable world knowledge (factuality)<n>textscINFACT evaluates models in four modes: Base (clean), Visual Degradation, Evidence, and Temporal Intervention for order-sensitive items.<n>Experiments on 14 representative Video-LLMs reveal that higher Base-mode accuracy does not reliably translate to higher reliability in the induced modes.
  arXiv  Detail & Related papers  (2026-03-12T03:03:16Z)
• Self-Aware Object Detection via Degradation Manifolds [3.8265249634979734]
  In safety-critical settings, it is insufficient to produce predictions without assessing whether the input remains within the detector's nominal operating regime.<n>We introduce a degradation-aware self-awareness framework based on degradation manifold.<n>Our method augments a standard detection backbone with a lightweight embedding head trained via contrastive learning.
  arXiv  Detail & Related papers  (2026-02-20T17:58:46Z)
• On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMs [15.301640007799735]
  We show that simple, controlled textual perturbations--misleading captions or incorrect chain-of-thought (CoT) traces--cause substantial drops in robustness and confidence.<n>To better understand these vulnerabilities, we analyze RL fine-tuning dynamics and uncover an accuracy-faithfulness trade-off.
  arXiv  Detail & Related papers  (2026-02-13T01:12:00Z)
• Illusions of Confidence? Diagnosing LLM Truthfulness via Neighborhood Consistency [78.91846841708586]
  We show that even facts answered with perfect self-consistency can rapidly collapse under mild contextual interference.<n>We propose Neighbor-Consistency Belief (NCB), a structural measure of belief that evaluates response coherence across a conceptual neighborhood.<n>We also present Structure-Aware Training (SAT), which optimize context-invariant belief structure and reduces long-tail knowledge brittleness by approximately 30%.
  arXiv  Detail & Related papers  (2026-01-09T16:23:21Z)
• DeLeaker: Dynamic Inference-Time Reweighting For Semantic Leakage Mitigation in Text-to-Image Models [55.30555646945055]
  Text-to-Image (T2I) models are vulnerable to semantic leakage.<n>We introduce DeLeaker, a lightweight approach that mitigates leakage by directly intervening on the model's attention maps.<n>SLIM is the first dataset dedicated to semantic leakage.
  arXiv  Detail & Related papers  (2025-10-16T17:39:21Z)
• TrustLoRA: Low-Rank Adaptation for Failure Detection under Out-of-distribution Data [62.22804234013273]
  We propose a simple failure detection framework to unify and facilitate classification with rejection under both covariate and semantic shifts.<n>Our key insight is that by separating and consolidating failure-specific reliability knowledge with low-rank adapters, we can enhance the failure detection ability effectively and flexibly.
  arXiv  Detail & Related papers  (2025-04-20T09:20:55Z)
• Robustness and Accuracy Could Be Reconcilable by (Proper) Definition [109.62614226793833]
  The trade-off between robustness and accuracy has been widely studied in the adversarial literature. We find that it may stem from the improperly defined robust error, which imposes an inductive bias of local invariance. By definition, SCORE facilitates the reconciliation between robustness and accuracy, while still handling the worst-case uncertainty.
  arXiv  Detail & Related papers  (2022-02-21T10:36:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.