2024-06-11 · NeurIPS · 312 citations · club pick

An Image is Worth 32 Tokens for Reconstruction and Generation

Qihang Yu, Mark Weber, Xueqing Deng, Xiaohui Shen, Daniel Cremers, Liang-Chieh Chen

Published at NeurIPS (the arXiv record still lists it as a preprint). 312 citations, 53 of them influential, as of the last refresh.

Abstract

Recent advancements in generative models have highlighted the crucial role of image tokenization in the efficient synthesis of high-resolution images. Tokenization, which transforms images into latent representations, reduces computational demands compared to directly processing pixels and enhances the effectiveness and efficiency of the generation process. Prior methods, such as VQGAN, typically utilize 2D latent grids with fixed downsampling factors. However, these 2D tokenizations face challenges in managing the inherent redundancies present in images, where adjacent regions frequently display similarities. To overcome this issue, we introduce Transformer-based 1-Dimensional Tokenizer (TiTok), an innovative approach that tokenizes images into 1D latent sequences. TiTok provides a more compact latent representation, yielding substantially more efficient and effective representations than conventional techniques. For example, a 256 x 256 x 3 image can be reduced to just 32 discrete tokens, a significant reduction from the 256 or 1024 tokens obtained by prior methods. Despite its compact nature, TiTok achieves competitive performance to state-of-the-art approaches. Specifically, using the same generator framework, TiTok attains 1.97 gFID, outperforming MaskGIT baseline significantly by 4.21 at ImageNet 256 x 256 benchmark. The advantages of TiTok become even more significant when it comes to higher resolution. At ImageNet 512 x 512 benchmark, TiTok not only outperforms state-of-the-art diffusion model DiT-XL/2 (gFID 2.74 vs. 3.04), but also reduces the image tokens by 64x, leading to 410x faster generation process. Our best-performing variant can significantly surpasses DiT-XL/2 (gFID 2.13 vs. 3.04) while still generating high-quality samples 74x faster.

arXiv comment: A compact 1D Image Tokenization method, leading to SOTA generation performance while being substantially faster. Project page at https://yucornetto.github.io/projects/titok.html

Ten-minute slide kit

Six slides is the whole talk: what was broken, what people tried, what these authors did, what the numbers say, where it falls over, and the sentence people should remember.

SLIDE 1

The problem

In recent years, image generation has experienced remarkable progress, driven by the significant advancements in both transformers \[19, 62, 66, 10, 67, [68\]](#page-13-1) and diffusion models \[16, 55, 29, 49, [21\]](#page-10-3). Mirroring the trends in generative language models \[48, [59\]](#page-12-5), the architecture of many contemporary image generation models incorporate a standard image tokenizer and de-tokenizer. This array of models utilizes tokenized image representations—ranging from continuous…

SLIDE 2

What came before

Images have been compressed since the early days of deep learning with autoencoders \[27, [63\]](#page-12-9). The general design of using an encoder that compresses high-dimensional images into a low-dimensional latent representation and then using a decoder to reverse the process, has proven to be successful over the years. Variational Autoencoders (VAEs) [\[34\]](#page-11-1) extend the paradigm by learning to map the input to a

SLIDE 3

The method

Not recoverable from the parsed text. Read this section in the paper yourself.

SLIDE 4

What they measured

Building upon TiTok, we explore a range of configurations, including the model size and the number of tokens, to identify the most efficient and effective setup for a 1D image tokenizer. These preliminary experiments serve to provide a thorough evaluation, seeking a practical configuration of TiTok. Unless specified otherwise, we train all models with images of resolution H = 256 and W = 256, using the open-source MaskGIT-VQGAN [\[9\]](#page-10-8) to supply proxy codes for training. The patch size for both…

SLIDE 5

Where it breaks

This paper proposes a novel 1D tokenization method designed to eliminate the fixed corresponding constraints of existing 2D tokenization methods. The 1D tokenization model is validated using the Vector Quantization (VQ) tokenizer formulation alongside a Masked Transformer generator framework. Despite the promising results, the proposed 1D tokenization formulation theoretically has the potential to generalize to other tokenizer formulations (*e.g*., 1D-VAE), other generation frameworks (*e.g*., Diffusion Models),…

SLIDE 6

One-line takeaway

Keynote 2 by by Arjun Subramaniam (Factory Intelligence)

Assembled from the paper's own PDF, parsed with its layout intact, 85,200 characters of it, then split on the paper's own section headings. Extractive, not generated: every sentence here is lifted from the paper. The takeaway line is the club's own one-liner from its reading list. Check it before you present it.

Figures worth putting on a slide

Figure 1: We propose TiTok, a compact 1D tokenizer leveraging region redundancy to represent an image with only 32 tokens for image reconstruction and generation.
Figure 2: A speed and quality comparison of TiTok and prior arts on ImageNet 256 × 256 and 512 × 512 generation benchmarks. Speed-up is compared against DiT-XL/2 [\[49\]](#page-12-
Figure 3: Illustration of image reconstruction (a) and generation (b) with the TiTok framework (c). TiTok contains an encoder Enc, a quantizer Quant, and a decoder Dec. Image patch
Figure 4: Preliminary experimental results with different TiTok variants. We provide a comprehensive exploration in (a) ImageNet-1K reconstruction. (b) ImageNet-1K linear probing.
Figure 5: Visualization of generated images from TiTok variants with MaskGIT [\[9\]](#page-10-8). Corresponding ImageNet class names are shown below the images.
Figure 6: Visualization of generated images from TiTok-L-32 with MaskGIT [\[9\]](#page-10-8) across random ImageNet classes.

Presented at

Saturday, May 23, 2026

Saturday Robotics, session 9

Robotics & World Models Reading Club 09: CVPR Warm-up & Founders Spotlight — DeltaWorld + VisuoTactile Dexterous Hands | San Francisco 0523

Listed on the event page as “A Frame is Worth One Token: Efficient Generative World Modeling with Delta Tokens”. The arXiv title above is the record.

San Francisco, CA

Why the club picked it. Keynote 2 by by Arjun Subramaniam (Factory Intelligence)