SNAP-UQ: Self-supervised Next-Activation Prediction for Single-Pass Uncertainty in TinyML

• URL: http://arxiv.org/abs/2508.12907v1
• Date: Mon, 18 Aug 2025 13:14:20 GMT
• Title: SNAP-UQ: Self-supervised Next-Activation Prediction for Single-Pass Uncertainty in TinyML
• Abstract summary: We introduce textbfSNAP-UQ, a single-pass, label-free uncertainty method for TinyML.<n>It estimates risk from emphdepth-wise next-activation prediction: tiny int8 heads forecast the statistics of the next layer from a compressed view of the previous one.<n>The design requires no temporal buffers, auxiliary exits, or repeated forward passes, and adds only a few tens of kilobytes to MCU deployments.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce \textbf{SNAP-UQ}, a single-pass, label-free uncertainty method for TinyML that estimates risk from \emph{depth-wise next-activation prediction}: tiny int8 heads forecast the statistics of the next layer from a compressed view of the previous one, and a lightweight monotone mapper turns the resulting surprisal into an actionable score. The design requires no temporal buffers, auxiliary exits, or repeated forward passes, and adds only a few tens of kilobytes to MCU deployments. Across vision and audio backbones, SNAP-UQ consistently reduces flash and latency relative to early-exit and deep ensembles (typically $\sim$40--60\% smaller and $\sim$25--35\% faster), with competing methods of similar accuracy often exceeding memory limits. In corrupted streams it improves accuracy-drop detection by several AUPRC points and maintains strong failure detection (AUROC $\approx$0.9) in a single pass. Grounding uncertainty in layer-to-layer dynamics yields a practical, resource-efficient basis for on-device monitoring in TinyML.

Related papers

• Prediction of Prediction (PoP): Inter-Layer Activation Fusion for Single-Pass Hallucination Detection in Large Language Models [0.0]
  This work evaluates whether internal hidden-state transition dynamics during generation can signal factual errors without auxiliary decoding calls.<n>We introduce Prediction of Prediction (PoP), a mechanism that captures layer-transition uncertainty.<n>PoP achieves an area under the receiver operating characteristic curve (AUROC) of 75.5% for factual-correctness classification.
  arXiv  Detail & Related papers  (2026-08-27T14:17:14Z)
• Adversarial Learning Game for Intrusion Detection in Quantum Key Distribution [0.0]
  This paper presents a high-fidelity simulation framework for intrusion detection in decoy-state Quantum Key Distribution (QKD)<n>The framework is modeled as a minimax game between a learning-based defender and an adaptive adversary.<n>Under adaptive attack scenarios, the system preserves $82text--92%$ of the honest finite-key rate while discarding only approximately $1.2%$ of traffic.
  arXiv  Detail & Related papers  (2026-03-03T20:21:06Z)
• FASA: Frequency-aware Sparse Attention [56.26881872333624]
  We propose FASA, a novel framework that achieves query-aware token eviction by dynamically predicting token importance.<n>Our key finding is that a small, identifiable subset of "dominant" FCs consistently exhibits high contextual agreement with the full attention head.<n>Across a spectrum of long-context tasks, FASA consistently outperforms all token-eviction baselines and achieves near-oracle accuracy.
  arXiv  Detail & Related papers  (2026-02-03T06:09:06Z)
• LUT-Compiled Kolmogorov-Arnold Networks for Lightweight DoS Detection on IoT Edge Devices [20.271194684947282]
  Kolmogorov-Arnold Networks (KANs) offer a compact alternative to Multi-Layer Perceptrons (MLPs)<n>B-spline evaluation introduces significant computational overhead unsuitable for latency-critical IoT applications.<n>We propose a lookup table (LUT) compilation pipeline that replaces expensive spline computations with precomputed quantized tables.
  arXiv  Detail & Related papers  (2026-01-12T22:32:18Z)
• TCUQ: Single-Pass Uncertainty Quantification from Temporal Consistency with Streaming Conformal Calibration for TinyML [0.0]
  TCUQ is a single pass, label free uncertainty monitor for streaming TinyML.<n>It converts short horizon temporal consistency captured via lightweight signals on posteriors.<n>A streaming conformal layer turns this score into a budgeted accept/abstain rule.
  arXiv  Detail & Related papers  (2025-08-18T13:12:14Z)
• R-Stitch: Dynamic Trajectory Stitching for Efficient Reasoning [80.104336426172]
  Chain-of-thought (CoT) enhances problem-solving ability of large language models.<n>CoT incurs substantial inference cost due to long autoregressive trajectories.<n>We introduce R-Stitch, a training-free hybrid decoding framework.
  arXiv  Detail & Related papers  (2025-07-23T08:14:36Z)
• Cautious Next Token Prediction [62.74127603725369]
  We propose a new training-free decoding strategy, dubbed as Cautious Next Token Prediction (CNTP)<n>In the decoding process, if the model has comparatively high prediction entropy at a certain step, we sample multiple trials starting from the step independently and stop when encountering any punctuation.<n>We show that our proposed CNTP approach outperforms existing standard decoding strategies consistently by a clear margin.
  arXiv  Detail & Related papers  (2025-07-03T05:49:18Z)
• $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts [55.231201692232894]
  $textttSPECS$ is a latency-aware test-time scaling method inspired by speculative decoding.<n>Our results show that $textttSPECS$matches or surpasses beam search accuracy while reducing latency by up to $sim$19.1%.
  arXiv  Detail & Related papers  (2025-06-15T05:50:05Z)
• Automatic mixed precision for optimizing gained time with constrained loss mean-squared-error based on model partition to sequential sub-graphs [0.8999666725996975]
  Mixed Precision (MP) mitigates the tradeoff by varying numerical precision across network layers.<n>This study focuses on automatically selecting an optimal MP configuration within Post-Training Quantization (PTQ) for inference.
  arXiv  Detail & Related papers  (2025-05-19T12:51:02Z)
• Mix-QSAM: Mixed-Precision Quantization of the Segment Anything Model [0.0]
  Mix-QSAM is a mixed-precision Post-Training Quantization (PTQ) framework for the Segment Anything Model (SAM)<n>We introduce a layer-wise importance score, derived using Kullback-Leibler (KL) divergence, to quantify each layer's contribution to the model's output.<n>We also introduce cross-layer synergy, a novel metric based on causal mutual information, to capture dependencies between adjacent layers.
  arXiv  Detail & Related papers  (2025-05-08T00:08:31Z)
• QUTE: Quantifying Uncertainty in TinyML with Early-exit-assisted ensembles for model-monitoring [0.0]
  Uncertainty quantification (UQ) provides a resource-efficient solution for on-device monitoring of tinyML models deployed without access to true labels. We propose QUTE, a novel resource-efficient early-exit-assisted ensemble architecture for optimized tinyML models. We show that QUTE delivers superior uncertainty quality on tiny models, achieving comparable performance on larger models with 59% smaller model sizes than the closest prior work.
  arXiv  Detail & Related papers  (2024-04-19T03:06:50Z)
• FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping [49.66872823080736]
  Autoregressive Large Language Models (e.g., LLaMa, GPTs) are omnipresent achieving remarkable success in language understanding and generation. To mitigate overload incurred during generation, several early-exit and layer-dropping strategies have been proposed. We propose FFN-SkipLLM, which is an input-adaptive feed-forward skipping strategy.
  arXiv  Detail & Related papers  (2024-04-05T02:35:43Z)
• Fast Minimum-norm Adversarial Attacks through Adaptive Norm Constraints [29.227720674726413]
  We propose a fast minimum-norm (FMN) attack that works with different $ell_p$-norm perturbation models. Experiments show that FMN significantly outperforms existing attacks in terms of convergence speed and time.
  arXiv  Detail & Related papers  (2021-02-25T12:56:26Z)
• Probing Model Signal-Awareness via Prediction-Preserving Input Minimization [67.62847721118142]
  We evaluate models' ability to capture the correct vulnerability signals to produce their predictions. We measure the signal awareness of models using a new metric we propose- Signal-aware Recall (SAR) The results show a sharp drop in the model's Recall from the high 90s to sub-60s with the new metric.
  arXiv  Detail & Related papers  (2020-11-25T20:05:23Z)
• Breaking the Sample Size Barrier in Model-Based Reinforcement Learning with a Generative Model [50.38446482252857]
  This paper is concerned with the sample efficiency of reinforcement learning, assuming access to a generative model (or simulator) We first consider $gamma$-discounted infinite-horizon Markov decision processes (MDPs) with state space $mathcalS$ and action space $mathcalA$. We prove that a plain model-based planning algorithm suffices to achieve minimax-optimal sample complexity given any target accuracy level.
  arXiv  Detail & Related papers  (2020-05-26T17:53:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.