Adversarially Robust Abductive Fusion of Pre-trained Transformer-based Perception Models

• URL: http://arxiv.org/abs/2608.04190v1
• Date: Tue, 04 Aug 2026 19:46:57 GMT
• Title: Adversarially Robust Abductive Fusion of Pre-trained Transformer-based Perception Models
• Abstract summary: Prior metacognitive methods learn logical rules that flag a model's errors, but rely on hand-authored domain-knowledge cues.<n>We show that this metacognitive layer can be learned without any domain knowledge by exploiting vector-space geometry.<n>Because the approach remains neurosymbolic, these rules share a single logical framework and can still be complemented by domain knowledge when available.
• Score: 5.574264082008515
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deploying pre-trained perception models in novel environments degrades their accuracy under distributional shift, and assembling them alone does not recover it: combiners such as majority voting trade recall for precision and are brittle to coordinated failures. Prior metacognitive methods learn logical rules that flag a model's errors, but rely on hand-authored domain-knowledge cues (object-size priors, segmentation masks) that do not transfer to genuinely novel scenes. We show that this metacognitive layer can be learned without any domain knowledge by exploiting vector-space geometry: per-model Label Vector Pools (LVP), built from each model's own training embeddings, yield error-detection rules from the geometry of detections relative to training-determined prototypes, reaching parity with domain-knowledge rules to within $0.002$ every F1 on test set. Because the approach remains neurosymbolic, these geometric rules share a single logical framework and can still be complemented by domain knowledge when available. We frame the fusion of multiple imperfect ViT-based detectors as a consistency-based abduction problem solved at test time by an exact Integer Program (IP) and a polynomial-time heuristic. On an aerial-imagery benchmark of 15 weather-shifted test sets and six ViT detectors, our domain-knowledge-free layer matches the strongest majority-vote variant on clean data (within $0.005$ F1) and, unlike every majority-vote baseline, retains its performance under a coordinated label-flipping attack: at a $90\%$ flip rate it averages $0.42$ F1 versus $0.35$ for MV-Plurality (a $22\%$ relative gain) and attains the highest F1 on \emph{every} test set once the flip rate exceeds $0.4$

Related papers

• Workload Identification with Physical Side Channels for AI Governance [0.5013248430919224]
  We show that an external observer can identify the class of the workload running on an NVIDIA H200 from its power draw.<n>Four evasion strategies are tested to disguise training as inference, producing an additional 680 adversarial traces.
  arXiv  Detail & Related papers  (2026-08-31T19:59:35Z)
• TRACE: A Multi-Layer Benchmark for Human AI Controller Coordination Under Drift and Failure [0.0]
  Modern cyber-physical and AI-assisted systems couple human operators, AI decision modules, and automated controllers in a single control loop.<n>No standard benchmark captures time-aligned, multi-layer traces of how drift and failures propagate across these layers.<n>We construct a benchmark by injecting controlled drift into traces derived from ALFRED, a grounded-instruction benchmark for everyday household tasks.
  arXiv  Detail & Related papers  (2026-08-07T00:03:55Z)
• Round-Trip Consistency: Bidirectional Diffusion Models Can Predict Their Own Rollout Errors [51.56484100374058]
  Autoregressive models accumulate error over long rollouts, yet at deployment there is no ground truth to measure it against.<n>We train a single conditional latent diffusion model that steps a dynamical system forward or backward in time via a direction flag.<n>We show that this bidirectionality supplies a measurement-free test-time error signal.
  arXiv  Detail & Related papers  (2026-08-01T13:49:46Z)
• AEGIS: Assay-Aware Protocol Validation and Runtime Monitoring for Open-Source Liquid Handling Robots [5.085084241476424]
  We present AEGIS, a two-layer guardian for open-source liquid handlers.<n>A curated machine-readable assay rule database with an LLM that reasons over OT-2 Python code.<n>Layer 2 fits a PCA world model to YOLO-cropped four-frame pipette trajectories.
  arXiv  Detail & Related papers  (2026-07-17T04:39:04Z)
• A Boundary-Metric Evaluation Protocol for Whiteboard Stroke Segmentation Under Extreme Imbalance [0.0]
  Boundary-aware metrics and a thin-subset equity analysis change how loss functions rank and exposes hidden trade-offs.<n>Five losses are trained three times each on a DeepLabV3-MobileNetV3 model and evaluated on 12 held-out images.<n>Overlap-based losses improve F1 by more than 20 points over cross-entropy.
  arXiv  Detail & Related papers  (2026-02-26T05:16:18Z)
• Beyond Accuracy: A Unified Random Matrix Theory Diagnostic Framework for Crash Classification Models [6.908972852063454]
  We introduce a diagnostic framework grounded in Random Matrix Theory (RMT) and Heavy-Tailed Self-Regularization (HTSR)<n>We evaluate nine model families on two Iowa DOT crash classification tasks (173,512 and 371,062 records respectively)<n>We find that the power-law exponent $$ provides a structural quality signal: well-regularized models consistently yield $$ within $[2, 4]$ (mean $2.87 pm 0.34$)<n>We propose an $$-based early stopping criterion and a spectral model selection protocol, and validate both against cross-validated F
  arXiv  Detail & Related papers  (2026-02-23T05:42:54Z)
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers [90.50039419576807]
  Reinforcement Learning with Verifiable Rewards (RLVR) trains policies against automated verifiers to avoid costly human labeling.<n>To reduce vulnerability to verifier hacking, many RLVR systems collapse rewards to binary $0,1$ during training.<n>This choice carries a cost: it introduces textitfalse negatives (rejecting correct answers, FNs) and textitfalse positives (accepting incorrect ones, FPs)
  arXiv  Detail & Related papers  (2025-10-01T13:56:44Z)
• Test time training enhances in-context learning of nonlinear functions [51.56484100374058]
  Test-time training (TTT) enhances model performance by explicitly updating designated parameters prior to each prediction.<n>We investigate the combination of TTT with in-context learning (ICL), where the model is given a few examples from the target distribution at inference time.
  arXiv  Detail & Related papers  (2025-09-30T03:56:44Z)
• MaskPro: Linear-Space Probabilistic Learning for Strict (N:M)-Sparsity on Large Language Models [53.36415620647177]
  Semi-structured sparsity offers a promising solution by strategically retaining $N$ elements out of every $M$ weights.<n>Existing (N:M)-compatible approaches typically fall into two categories: rule-based layerwise greedy search, which suffers from considerable errors, and gradient-driven learning, which incurs prohibitive training costs.<n>We propose a novel linear-space probabilistic framework named MaskPro, which aims to learn a prior categorical distribution for every $M$ consecutive weights and subsequently leverages this distribution to generate the (N:M)-sparsity throughout an $N$-way sampling
  arXiv  Detail & Related papers  (2025-06-15T15:02:59Z)
• Exact Certification of (Graph) Neural Networks Against Label Poisoning [50.87615167799367]
  We introduce an exact certification method for label flipping in Graph Neural Networks (GNNs)<n>We apply our method to certify a broad range of GNN architectures in node classification tasks.<n>Our work presents the first exact certificate to a poisoning attack ever derived for neural networks.
  arXiv  Detail & Related papers  (2024-11-30T17:05:12Z)
• SimAD: A Simple Dissimilarity-based Approach for Time Series Anomaly Detection [23.684577046512747]
  We introduce a $textbfSim$ple dissimilarity-based approach for time series $textbfA$nomaly $textbfD$etection, referred to as $textbfSimAD$.<n>SimAD first incorporates a patching-based feature extractor capable of processing extended temporal windows and employs the EmbedPatch encoder to fully integrate normal behavioral patterns.<n>Second, we design an innovative ContrastFusion module in SimAD, which strengthens the robustness of anomaly detection by highlighting the distributional differences between normal and abnormal data.
  arXiv  Detail & Related papers  (2024-05-18T09:37:04Z)
• Cross-Model Pseudo-Labeling for Semi-Supervised Action Recognition [98.25592165484737]
  We propose a more effective pseudo-labeling scheme, called Cross-Model Pseudo-Labeling (CMPL) CMPL achieves $17.6%$ and $25.1%$ Top-1 accuracy on Kinetics-400 and UCF-101 using only the RGB modality and $1%$ labeled data, respectively.
  arXiv  Detail & Related papers  (2021-12-17T18:59:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.