ByteAction: Byte-space Action Recognition Foundation Model

• URL: http://arxiv.org/abs/2608.22760v1
• Date: Mon, 24 Aug 2026 03:31:51 GMT
• Title: ByteAction: Byte-space Action Recognition Foundation Model
• Abstract summary: Byte-space Action Recognition (BAR) aims to recognize human actions directly from compressed image bitstreams without any pixel decoding.<n>We propose ByteAction, a BAR foundation model that achieves accurate action recognition on corrupted image bitstreams.
• Score: 21.295794335147082
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Byte-space Action Recognition (BAR) aims to recognize human actions directly from compressed image bitstreams without any pixel decoding. By operating entirely in byte space, BAR is inherently independent of file integrity and pixel-level reconstruction, making it naturally applicable to privacy-sensitive scenarios and robust against bitstream corruption. In this paper, we propose ByteAction, a BAR foundation model that achieves accurate action recognition on corrupted image bitstreams. ByteAction follows a dual-view byte-level recognition framework. It constructs weakly and strongly corrupted bitstream views, which are augmented by Bitstream Pattern Augmentation (BPA) and encoded with a shared ByteFormer backbone. The model is optimized with both classification and corruption consistency objectives. Specifically, we propose Bitstream Pattern Augmentation (BPA), which reshapes one-dimensional byte sequences into two-dimensional byte matrix and applies region-level erasure to encourage the model to learn robust cross-region byte dependencies. We further propose a Corruption Consistency Training strategy that constrains the model to maintain stable predictions across different corruption severities through bidirectional KL divergence. Experiments on the image bitstream from Stanford40, PPMI, and PASCAL VOC 2012 Action demonstrate that ByteAction achieves state-of-the-art corruption robustness across all scenarios while maintaining competitive intact bitstream performance.

Related papers

• Towards Bitstream-corrupted Harsh Visual Understanding: Through Bitstream Language Modeling as Robust Semantic Priors [14.91030843318295]
  Bitstream-corrupted Harsh Visual Understanding (BcHVU) aims to understand harshly degraded videos originally decoded from a severely corrupted bitstream in real-world multimedia communication.<n>We propose Bitstream Language Modeling as Robust Semantic Priors (BLMSP), a framework for learning and injecting bitstream-native semantic cues.<n>Our proposed BLMSP framework learns to extract bitstream-native semantic cues by bitstream language modeling, and leverages them as priors by injecting into off-the-shelf vision models of BcHVU tasks.
  arXiv  Detail & Related papers  (2026-08-22T08:16:58Z)
• Bitstream Action Recognition is Byte Modeling [33.87306245425509]
  Bitstream corruption during storage or transmission may cause severe visual artifacts or even decoding failure.<n>We propose a novel BAR framework, Bitstream Recognition via Anchoring Corrupted Embeddings (BRACE)<n>BRACE is a dual-branch byte-modeling architecture that treats a corrupted bitstream and its intact counterpart as two byte realizations of the same action.
  arXiv  Detail & Related papers  (2026-08-16T11:57:05Z)
• Robust Promptable Video Object Segmentation [67.1533741758339]
  This paper offers the first comprehensive study on robust PVOS (RobustPVOS)<n>We first construct a new, comprehensive benchmark with two real-world evaluation datasets of 351 video clips and more than 2,500 object masks.<n>We present a new RobustPVOS method, dubbed Memory-object-conditioned Gated-rank Adaptation (MoGA)
  arXiv  Detail & Related papers  (2026-05-12T11:55:31Z)
• Bitstream Collisions in Neural Image Compression via Adversarial Perturbations [2.0960189135529212]
  This study reveals an unexpected vulnerability in NIC - bitstream collisions.<n>The collision vulnerability poses a threat to the practical usability of NIC, particularly in security-critical applications.<n>A simple yet effective mitigation method is presented.
  arXiv  Detail & Related papers  (2025-03-25T16:29:17Z)
• ByteNet: Rethinking Multimedia File Fragment Classification through Visual Perspectives [23.580848165023962]
  Multimedia file fragment classification (MFFC) aims to identify file fragment types without system metadata. Existing MFFC methods treat fragments as 1D byte sequences and emphasize the relations between separate bytes (interbytes) for classification. Byte2Image incorporates previously overlooked intrabyte information into file fragments and reinterprets these fragments as 2D images. ByteNet makes full use of the raw 1D byte sequence and the converted 2D image through a shallow byte branch feature extraction (BBFE) and a deep image branch feature extraction (IBFE) network.
  arXiv  Detail & Related papers  (2024-10-28T09:19:28Z)
• HybridFlow: Infusing Continuity into Masked Codebook for Extreme Low-Bitrate Image Compression [51.04820313355164]
  HyrbidFlow combines the continuous-feature-based and codebook-based streams to achieve both high perceptual quality and high fidelity under extreme lows. Experimental results demonstrate superior performance across several datasets under extremely lows.
  arXiv  Detail & Related papers  (2024-04-20T13:19:08Z)
• IBVC: Interpolation-driven B-frame Video Compression [68.18440522300536]
  B-frame video compression aims to adopt bi-directional motion estimation and motion compensation (MEMC) coding for middle frame reconstruction. Previous learned approaches often directly extend neural P-frame codecs to B-frame relying on bi-directional optical-flow estimation. We propose a simple yet effective structure called Interpolation-B-frame Video Compression (IBVC) to address these issues.
  arXiv  Detail & Related papers  (2023-09-25T02:45:51Z)
• Enhancing Infrared Small Target Detection Robustness with Bi-Level Adversarial Framework [61.34862133870934]
  We propose a bi-level adversarial framework to promote the robustness of detection in the presence of distinct corruptions. Our scheme remarkably improves 21.96% IOU across a wide array of corruptions and notably promotes 4.97% IOU on the general benchmark.
  arXiv  Detail & Related papers  (2023-09-03T06:35:07Z)
• You Can Mask More For Extremely Low-Bitrate Image Compression [80.7692466922499]
  Learned image compression (LIC) methods have experienced significant progress during recent years. LIC methods fail to explicitly explore the image structure and texture components crucial for image compression. We present DA-Mask that samples visible patches based on the structure and texture of original images. We propose a simple yet effective masked compression model (MCM), the first framework that unifies LIC and LIC end-to-end for extremely low-bitrate compression.
  arXiv  Detail & Related papers  (2023-06-27T15:36:22Z)
• Bytes Are All You Need: Transformers Operating Directly On File Bytes [55.81123238702553]
  We investigate modality-independent representation learning by performing classification on file bytes, without the need for decoding files at inference time. Our model, ByteFormer, improves ImageNet Top-1 classification accuracy by $5%$. We demonstrate that the same ByteFormer architecture can perform audio classification without modifications or modality-specific preprocessing.
  arXiv  Detail & Related papers  (2023-05-31T23:18:21Z)
• Expressive Losses for Verified Robustness via Convex Combinations [67.54357965665676]
  We study the relationship between the over-approximation coefficient and performance profiles across different expressive losses. We show that, while expressivity is essential, better approximations of the worst-case loss are not necessarily linked to superior robustness-accuracy trade-offs.
  arXiv  Detail & Related papers  (2023-05-23T12:20:29Z)
• A Byte Sequence is Worth an Image: CNN for File Fragment Classification Using Bit Shift and n-Gram Embeddings [21.14735408046021]
  File fragment classification (FFC) on small chunks of memory is essential in memory forensics and Internet security. Existing methods mainly treat file fragments as 1d byte signals and utilize the captured inter-byte features for classification. We propose Byte2Image, a novel data augmentation technique, to introduce the neglected intra-byte information into file fragments and re-treat them as 2d gray-scale images.
  arXiv  Detail & Related papers  (2023-04-14T08:06:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.