Towards Defining an Efficient and Expandable File Format for AI-Generated Contents

• URL: http://arxiv.org/abs/2410.09834v3
• Date: Sun, 09 Feb 2025 13:35:45 GMT
• Title: Towards Defining an Efficient and Expandable File Format for AI-Generated Contents
• Authors: Yixin Gao, Runsen Feng, Xin Li, Weiping Li, Zhibo Chen,
• Abstract summary: We propose a new file format for AIGC images, named AIGIF, enabling ultra-low coding of AIGC images.<n>We experimentally find that a well-designed composable bitstream structure incorporating the above three factors can achieve an impressive compression ratio of even up to 1/10,000.
• Score: 23.217964968742823
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recently, AI-generated content (AIGC) has gained significant traction due to its powerful creation capability. However, the storage and transmission of large amounts of high-quality AIGC images inevitably pose new challenges for recent file formats. To overcome this, we define a new file format for AIGC images, named AIGIF, enabling ultra-low bitrate coding of AIGC images. Unlike compressing AIGC images intuitively with pixel-wise space as existing file formats, AIGIF instead compresses the generation syntax. This raises a crucial question: Which generation syntax elements, e.g., text prompt, device configuration, etc, are necessary for compression/transmission? To answer this question, we systematically investigate the effects of three essential factors: platform, generative model, and data configuration. We experimentally find that a well-designed composable bitstream structure incorporating the above three factors can achieve an impressive compression ratio of even up to 1/10,000 while still ensuring high fidelity. We also introduce an expandable syntax in AIGIF to support the extension of the most advanced generation models to be developed in the future.

Related papers

• Why Compress What You Can Generate? When GPT-4o Generation Ushers in Image Compression Fields [14.805239427360208]
  AIGC foundation models are powerful enough to faithfully generate intricate structure and fine-grained details from nothing more than compact descriptors. Recent GPT-4o image generation of OpenAI has achieved impressive cross-modality generation, editing, and design capabilities.
  arXiv  Detail & Related papers  (2025-04-30T17:20:14Z)
• HiTVideo: Hierarchical Tokenizers for Enhancing Text-to-Video Generation with Autoregressive Large Language Models [63.65066762436074]
  HiTVideo aims to address the potential limitations of existing video tokenizers in text-to-video generation tasks. It utilizes a 3D causal VAE with a multi-layer discrete token framework, encoding video content into hierarchically structured codebooks.
  arXiv  Detail & Related papers  (2025-03-14T15:36:39Z)
• Stable Diffusion is a Natural Cross-Modal Decoder for Layered AI-generated Image Compression [7.643300240138419]
  We introduce a scalable cross-modal compression framework that incorporates multiple human-comprehensible modalities. Our framework encodes images into a layered bitstream consisting of a semantic layer that delivers high-level semantic information. Our method proficiently restores both semantic and visual details, competing against baseline approaches at extremely lows.
  arXiv  Detail & Related papers  (2024-12-17T15:01:35Z)
• PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision-Language Models [64.9366388601049]
  Visual token compression is leveraged to reduce the considerable token length of visual inputs. We introduce a unified token compression strategy called Progressive Visual Token Compression. Our model achieves state-of-the-art performance across various video understanding benchmarks.
  arXiv  Detail & Related papers  (2024-12-12T18:59:40Z)
• ImageFolder: Autoregressive Image Generation with Folded Tokens [51.815319504939396]
  Increasing token length is a common approach to improve the image reconstruction quality. There exists a trade-off between reconstruction and generation quality regarding token length. We propose Image, a semantic tokenizer that provides spatially aligned image tokens that can be folded during autoregressive modeling.
  arXiv  Detail & Related papers  (2024-10-02T17:06:39Z)
• Tell Codec What Worth Compressing: Semantically Disentangled Image Coding for Machine with LMMs [47.7670923159071]
  We present a new image compression paradigm to achieve intelligently coding for machine'' by cleverly leveraging the common sense of Large Multimodal Models (LMMs) We dub our method textitSDComp'' for textitSemantically textitDisentangled textitCompression'', and compare it with state-of-the-art codecs on a wide variety of different vision tasks.
  arXiv  Detail & Related papers  (2024-08-16T07:23:18Z)
• CE-VAE: Capsule Enhanced Variational AutoEncoder for Underwater Image Enhancement [8.16306466526838]
  Unmanned underwater image analysis for marine monitoring faces two key challenges: degraded image quality and hardware storage constraints. We introduce the Capsule Enhanced Variational AutoEncoder (CE-VAE), a novel architecture designed to efficiently compress and enhance degraded underwater images. CE-VAE achieves state-of-the-art performance in underwater image enhancement on six benchmark datasets.
  arXiv  Detail & Related papers  (2024-06-03T13:04:42Z)
• MISC: Ultra-low Bitrate Image Semantic Compression Driven by Large Multimodal Model [78.4051835615796]
  This paper proposes a method called Multimodal Image Semantic Compression. It consists of an LMM encoder for extracting the semantic information of the image, a map encoder to locate the region corresponding to the semantic, an image encoder generates an extremely compressed bitstream, and a decoder reconstructs the image based on the above information. It can achieve optimal consistency and perception results while saving perceptual 50%, which has strong potential applications in the next generation of storage and communication.
  arXiv  Detail & Related papers  (2024-02-26T17:11:11Z)
• Any-Size-Diffusion: Toward Efficient Text-Driven Synthesis for Any-Size HD Images [56.17404812357676]
  Stable diffusion, a generative model used in text-to-image synthesis, frequently encounters composition problems when generating images of varying sizes. We propose a two-stage pipeline named Any-Size-Diffusion (ASD), designed to efficiently generate well-composed images of any size. We show that ASD can produce well-structured images of arbitrary sizes, cutting down the inference time by 2x compared to the traditional tiled algorithm.
  arXiv  Detail & Related papers  (2023-08-31T09:27:56Z)
• GIT: A Generative Image-to-text Transformer for Vision and Language [138.91581326369837]
  We train a Generative Image-to-text Transformer, GIT, to unify vision-language tasks such as image/video captioning and question answering. Our model surpasses the human performance for the first time on TextCaps (138.2 vs. 125.5 in CIDEr)
  arXiv  Detail & Related papers  (2022-05-27T17:03:38Z)
• A Shared Representation for Photorealistic Driving Simulators [83.5985178314263]
  We propose to improve the quality of generated images by rethinking the discriminator architecture. The focus is on the class of problems where images are generated given semantic inputs, such as scene segmentation maps or human body poses. We aim to learn a shared latent representation that encodes enough information to jointly do semantic segmentation, content reconstruction, along with a coarse-to-fine grained adversarial reasoning.
  arXiv  Detail & Related papers  (2021-12-09T18:59:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.