FuguReport
Daily Report / 2026-06-30

Daily Paper Briefing

2026-06-30 1 Paper

GEAR: Guided End-to-End AutoRegression for Image Synthesis

Authors Bin Lin, Zheyuan Liu, Chenguo Lin, Sixiang Chen, Yunyang Ge, Yunlong Lin, Jianwei Zhang, Miles Yang, Zhao Zhong, Liefeng Bo, Li Yuan
Affiliations Peking University / Tencent
Categories Method / Image Synthesis / Guided end-to-end autoregressive training, Evaluation / Model Evaluation / ImageNet fFID convergence speed, Method / Tokenization / Vector quantization for image tokens
License CC BY 4.0

Abstract Overview

This paper introduces GEAR, a method for jointly training a vector-quantized tokenizer and an autoregressive image generator instead of freezing the tokenizer before generator training. The core challenge is that discrete VQ indices block gradients from the autoregressive model to the tokenizer, and the paper reports that a naive straight-through approach is unstable and can collapse. GEAR addresses this with a dual read-out: a hard branch for standard next-token prediction and a differentiable soft branch that sends a representation-alignment signal back only to the tokenizer. Experiments on class-conditional ImageNet and text-to-image generation indicate faster convergence, improved generation quality, and transfer across multiple quantizers.

Novelty

The distinctive contribution is an end-to-end training scheme for discrete VQ autoregressive generation that avoids sending prediction-loss gradients through non-differentiable token assignments. Its dual hard/soft assignment design uses representation alignment as the guidance channel, shifting the burden of semantic alignment from the tokenizer to the autoregressive model.

Results

On ImageNet, GEAR is reported to speed up gFID convergence by up to 10x relative to LlamaGen-REPA and to improve final generation metrics across B/L/XL model scales. In controlled text-to-image experiments on GPIC, it consistently outperforms the same baseline at matched training budgets, and ablations show gains across VQVAE, LFQ, and IBQ tokenizers while the straight-through variant collapses.

Key Points

  1. GEAR trains the VQ tokenizer and autoregressive generator jointly using a hard branch for next-token prediction and a soft differentiable branch for tokenizer guidance.
  2. The method improves convergence and final generation quality on class-conditional ImageNet and also shows consistent gains in controlled text-to-image experiments.
  3. Representation analysis suggests the tokenizer becomes less DINOv2-like while the autoregressive model becomes more aligned at the patch level, indicating that end-to-end guidance mainly improves local predictive structure in the generator.

References

GitHub Project
This page was created using generative AI such as GPT-5, Claude Opus 4, Gemini 3, Gemini 3.1 Flash Image, and their higher-end successor versions. No guarantee can be made regarding its contents.