Circuit Complexity Bounds for Visual Autoregressive Model

• URL: http://arxiv.org/abs/2501.04299v1
• Date: Wed, 08 Jan 2025 06:07:33 GMT
• Title: Circuit Complexity Bounds for Visual Autoregressive Model
• Authors: Yekun Ke, Xiaoyu Li, Yingyu Liang, Zhenmei Shi, Zhao Song,
• Abstract summary: We investigate the circuit complexity of the Visual AutoRegressive ( VAR) model and establish a bound in this study.<n>Our primary result demonstrates that the VAR model is equivalent to a simulation by a uniform $mathsfTC0$ threshold circuit with hidden dimension $d leq O(n)$ and $mathrmpoly(n)$ precision.<n>This is the first study to rigorously highlight the limitations in the expressive power of VAR models despite their impressive performance.
• Score: 23.991344681741058
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding the expressive ability of a specific model is essential for grasping its capacity limitations. Recently, several studies have established circuit complexity bounds for Transformer architecture. Besides, the Visual AutoRegressive (VAR) model has risen to be a prominent method in the field of image generation, outperforming previous techniques, such as Diffusion Transformers, in generating high-quality images. We investigate the circuit complexity of the VAR model and establish a bound in this study. Our primary result demonstrates that the VAR model is equivalent to a simulation by a uniform $\mathsf{TC}^0$ threshold circuit with hidden dimension $d \leq O(n)$ and $\mathrm{poly}(n)$ precision. This is the first study to rigorously highlight the limitations in the expressive power of VAR models despite their impressive performance. We believe our findings will offer valuable insights into the inherent constraints of these models and guide the development of more efficient and expressive architectures in the future.

Related papers

• Proxy-Tuning: Tailoring Multimodal Autoregressive Models for Subject-Driven Image Generation [24.67443650398078]
  We introduce Proxy-Tuning, leveraging diffusion models to enhance AR models' capabilities in subject-specific image generation. Our method reveals a striking weak-to-strong phenomenon: fine-tuned AR models consistently outperform their diffusion model supervisors in both subject fidelity and prompt adherence.
  arXiv  Detail & Related papers  (2025-03-13T07:32:57Z)
• On Computational Limits of FlowAR Models: Expressivity and Efficiency [37.56422161656334]
  FlowAR is a state-of-the-art architecture that integrates flow-based and autoregressive mechanisms. We show that when the largest feature map produced by the FlowAR model has dimensions $n times n times c$, the FlowAR model is simulable by a family of threshold circuits. We also identify the conditions under which the FlowAR model computations can achieve almost quadratic time.
  arXiv  Detail & Related papers  (2025-02-23T08:07:35Z)
• Jet: A Modern Transformer-Based Normalizing Flow [62.2573739835562]
  We revisit the design of the coupling-based normalizing flow models by carefully ablating prior design choices.<n>We achieve state-of-the-art quantitative and qualitative performance with a much simpler architecture.
  arXiv  Detail & Related papers  (2024-12-19T18:09:42Z)
• Solving Inverse Problems with Model Mismatch using Untrained Neural Networks within Model-based Architectures [14.551812310439004]
  We introduce an untrained forward model residual block within the model-based architecture to match the data consistency in the measurement domain for each instance. Our approach offers a unified solution that is less parameter-sensitive, requires no additional data, and enables simultaneous fitting of the forward model and reconstruction in a single pass.
  arXiv  Detail & Related papers  (2024-03-07T19:02:13Z)
• Converting Transformers to Polynomial Form for Secure Inference Over Homomorphic Encryption [45.00129952368691]
  Homomorphic Encryption (HE) has emerged as one of the most promising approaches in deep learning. We introduce the first transformer, providing the first demonstration of secure inference over HE with transformers. Our models yield results comparable to traditional methods, bridging the performance gap with transformers of similar scale and underscoring the viability of HE for state-of-the-art applications.
  arXiv  Detail & Related papers  (2023-11-15T00:23:58Z)
• Dissecting Multimodality in VideoQA Transformer Models by Impairing Modality Fusion [54.33764537135906]
  VideoQA Transformer models demonstrate competitive performance on standard benchmarks. Do these models capture the rich multimodal structures and dynamics from video and text jointly? Are they achieving high scores by exploiting biases and spurious features?
  arXiv  Detail & Related papers  (2023-06-15T06:45:46Z)
• Learning Versatile 3D Shape Generation with Improved AR Models [91.87115744375052]
  Auto-regressive (AR) models have achieved impressive results in 2D image generation by modeling joint distributions in the grid space. We propose the Improved Auto-regressive Model (ImAM) for 3D shape generation, which applies discrete representation learning based on a latent vector instead of volumetric grids.
  arXiv  Detail & Related papers  (2023-03-26T12:03:18Z)
• Closed-form Continuous-Depth Models [99.40335716948101]
  Continuous-depth neural models rely on advanced numerical differential equation solvers. We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
  arXiv  Detail & Related papers  (2021-06-25T22:08:51Z)
• Decision Transformer: Reinforcement Learning via Sequence Modeling [102.86873656751489]
  We present a framework that abstracts Reinforcement Learning (RL) as a sequence modeling problem. We present Decision Transformer, an architecture that casts the problem of RL as conditional sequence modeling. Despite its simplicity, Decision Transformer matches or exceeds the performance of state-of-the-art offline RL baselines on Atari, OpenAI Gym, and Key-to-Door tasks.
  arXiv  Detail & Related papers  (2021-06-02T17:53:39Z)
• Normalizing Flows with Multi-Scale Autoregressive Priors [131.895570212956]
  We introduce channel-wise dependencies in their latent space through multi-scale autoregressive priors (mAR) Our mAR prior for models with split coupling flow layers (mAR-SCF) can better capture dependencies in complex multimodal data. We show that mAR-SCF allows for improved image generation quality, with gains in FID and Inception scores compared to state-of-the-art flow-based models.
  arXiv  Detail & Related papers  (2020-04-08T09:07:11Z)
• Hybrid modeling: Applications in real-time diagnosis [64.5040763067757]
  We outline a novel hybrid modeling approach that combines machine learning inspired models and physics-based models. We are using such models for real-time diagnosis applications.
  arXiv  Detail & Related papers  (2020-03-04T00:44:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.