Bitstream Action Recognition is Byte Modeling

• URL: http://arxiv.org/abs/2608.15695v1
• Date: Sun, 16 Aug 2026 11:57:05 GMT
• Title: Bitstream Action Recognition is Byte Modeling
• Abstract summary: Bitstream corruption during storage or transmission may cause severe visual artifacts or even decoding failure.<n>We propose a novel BAR framework, Bitstream Recognition via Anchoring Corrupted Embeddings (BRACE)<n>BRACE is a dual-branch byte-modeling architecture that treats a corrupted bitstream and its intact counterpart as two byte realizations of the same action.
• Score: 33.87306245425509
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conventional action recognition typically relies on successful pixel decoding of the bitstream. However, bitstream corruption during storage or transmission may cause severe visual artifacts or even decoding failure, posing a significant challenge to reliable action recognition. Bitstream Action Recognition (BAR) aims to overcome the dependency on decoding and the vulnerability to corruption. In this paper, we propose a novel BAR framework, Bitstream Recognition via Anchoring Corrupted Embeddings (BRACE). BRACE is a dual-branch byte-modeling architecture that treats a corrupted bitstream and its intact counterpart as two byte realizations of the same action. This guides the generation of rich and stable representations for robustness to corruption through Intact-Anchored Representation Alignment (IARA). The intact representation serves as a stable anchor, and the corrupted one is aligned to it at the embedding and decision levels under Unreliable-Anchor Suppression (UAS), entirely in representation space and without repairing the bitstream. To address the scarcity of corrupted bitstreams in practice, we introduce the Real-world Bitstream Corruption Simulator (RBCS), a four-parameter simulator that reproduces bit-flip and byte-loss errors arising in transmission and storage. Building on RBCS, we construct the first large-scale BAR dataset (BAR-D), which comprises the BAR-Stanford40 and BAR-PPMI subsets and spans diverse corruption types and severity levels. Finally, we build a large benchmark on BAR-D involving 14 action recognition methods from the pixel, compressed, and bitstream domains. Extensive experiments demonstrate that BRACE has superior robustness to bitstream corruption than all comparison methods. Ablation studies further validate the effectiveness of the proposed RBCS augmentation and IARA.

Related papers

• ByteAction: Byte-space Action Recognition Foundation Model [21.295794335147082]
  Byte-space Action Recognition (BAR) aims to recognize human actions directly from compressed image bitstreams without any pixel decoding.<n>We propose ByteAction, a BAR foundation model that achieves accurate action recognition on corrupted image bitstreams.
  arXiv  Detail & Related papers  (2026-08-24T03:31:51Z)
• Towards Bitstream-corrupted Harsh Visual Understanding: Through Bitstream Language Modeling as Robust Semantic Priors [14.91030843318295]
  Bitstream-corrupted Harsh Visual Understanding (BcHVU) aims to understand harshly degraded videos originally decoded from a severely corrupted bitstream in real-world multimedia communication.<n>We propose Bitstream Language Modeling as Robust Semantic Priors (BLMSP), a framework for learning and injecting bitstream-native semantic cues.<n>Our proposed BLMSP framework learns to extract bitstream-native semantic cues by bitstream language modeling, and leverages them as priors by injecting into off-the-shelf vision models of BcHVU tasks.
  arXiv  Detail & Related papers  (2026-08-22T08:16:58Z)
• Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokens [22.687236570529844]
  We propose ASRD (Anchor Supervised Revocable Decoding), a training-free framework that operates within the embedding space.<n>We show that ASRD outperforms recent remasking baselines, achieving accuracy improvements of up to 6.4% while accelerating inference throughput by up to 7.2times$.
  arXiv  Detail & Related papers  (2026-06-15T15:23:47Z)
• Edit the Bits, Diff the Codes: Bitwise Residual Editing for Visual Autoregressive Models [85.59447229497101]
  BitResEdit is a training-free editor for bitwise-residual VAR generators such as Infinity.<n>It performs source-negative guidance by tilting the post-CFG per-bit log-odds along a source--target contrast computed on a shared edited prefix.<n>It converts the sampled bits into per-scale continuous-code residuals, gates them with a localization mask, and re-injects them through the generator's native sum-of-scales.
  arXiv  Detail & Related papers  (2026-06-11T16:41:25Z)
• BitC-3DGS: High-Capacity 3D Gaussian Splatting Watermarking via Bit Compression [68.33622936663876]
  High-capacity watermarking is necessary for 3D Gaussian Splatting (3DGS) assets to embed rich information.<n>Existing bit-to-token watermarking methods are limited to 77-bit messages due to CLIP's fixed 77-token context length.<n>We introduce BitC-3DGS, a bit-compression framework that encodes multiple message bits per token.
  arXiv  Detail & Related papers  (2026-05-28T08:28:04Z)
• An Application-Layer Multi-Modal Covert-Channel Reference Monitor for LLM Agent Egress [0.0]
  A large language model (LLM) agent that sends messages can leak data inside them.<n>A compromised agent encodes bits in zero-width characters, homoglyphs, whitespace, base64, JavaScript Object Notation (JSON) key ordering, message timing or size.<n>Our egress reference monitor has three contributions.
  arXiv  Detail & Related papers  (2026-05-20T05:39:29Z)
• Blind Bitstream-corrupted Video Recovery via Metadata-guided Diffusion Model [21.33395654187404]
  We introduce a new blind video recovery setting that removes the reliance on predefined masks.<n>We propose a Metadata-Guided Diffusion Model (M-GDM) to tackle these challenges.
  arXiv  Detail & Related papers  (2026-04-15T14:15:07Z)
• Transformers from Compressed Representations [74.48571451824569]
  TEMPEST (TransformErs froM comPressed rEpreSenTations) is a method that exploits the inherent byte-stream structure of compressed files to design an effective tokenization and encoding strategy.<n>Our proposal substantially reduces the number of tokens required for semantic classification, thereby lowering both computational complexity and memory usage.
  arXiv  Detail & Related papers  (2025-10-26T13:48:03Z)
• HybridFlow: Infusing Continuity into Masked Codebook for Extreme Low-Bitrate Image Compression [51.04820313355164]
  HyrbidFlow combines the continuous-feature-based and codebook-based streams to achieve both high perceptual quality and high fidelity under extreme lows. Experimental results demonstrate superior performance across several datasets under extremely lows.
  arXiv  Detail & Related papers  (2024-04-20T13:19:08Z)
• You Can Mask More For Extremely Low-Bitrate Image Compression [80.7692466922499]
  Learned image compression (LIC) methods have experienced significant progress during recent years. LIC methods fail to explicitly explore the image structure and texture components crucial for image compression. We present DA-Mask that samples visible patches based on the structure and texture of original images. We propose a simple yet effective masked compression model (MCM), the first framework that unifies LIC and LIC end-to-end for extremely low-bitrate compression.
  arXiv  Detail & Related papers  (2023-06-27T15:36:22Z)
• Bitstream-Corrupted JPEG Images are Restorable: Two-stage Compensation and Alignment Framework for Image Restoration [23.101965117344]
  We study a real-world JPEG image restoration problem with bit errors on the encrypted bitstream. We propose a robust JPEG decoder, followed by a two-stage compensation and alignment framework. We conduct experiments on three benchmarks with varying degrees of bit error rates.
  arXiv  Detail & Related papers  (2023-04-14T07:54:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.