2023-01-10 · 1,333 citations · club pick

Mastering Diverse Domains through World Models

Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, Timothy Lillicrap

No peer-reviewed venue on record yet. 1,333 citations, 179 of them influential, as of the last refresh.

Abstract

Developing a general algorithm that learns to solve tasks across a wide range of applications has been a fundamental challenge in artificial intelligence. Although current reinforcement learning algorithms can be readily applied to tasks similar to what they have been developed for, configuring them for new application domains requires significant human expertise and experimentation. We present DreamerV3, a general algorithm that outperforms specialized methods across over 150 diverse tasks, with a single configuration. Dreamer learns a model of the environment and improves its behavior by imagining future scenarios. Robustness techniques based on normalization, balancing, and transformations enable stable learning across domains. Applied out of the box, Dreamer is the first algorithm to collect diamonds in Minecraft from scratch without human data or curricula. This achievement has been posed as a significant challenge in artificial intelligence that requires exploring farsighted strategies from pixels and sparse rewards in an open world. Our work allows solving challenging control problems without extensive experimentation, making reinforcement learning broadly applicable.

arXiv comment: Website: https://danijar.com/dreamerv3

Ten-minute slide kit

Six slides is the whole talk: what was broken, what people tried, what these authors did, what the numbers say, where it falls over, and the sentence people should remember.

SLIDE 1

The problem

Reinforcement learning has enabled computers to solve tasks through interaction, such as surpassing humans in the games of Go and Dota1,2 . It is also a key component for improving large language models beyond what is demonstrated in their pretraining data 3,4 . While PPO5 has become a standard algorithm in the field of reinforcement learning, more specialized algorithms are often employed to achieve higher

SLIDE 2

What came before

Not recoverable from the parsed text. Read this section in the paper yourself.

SLIDE 3

The method

Danijar Hafner,12 Jurgis Pasukonis,1 Jimmy Ba,2 Timothy Lillicrap1 The world model learns compact representations of sensory inputs through autoencoding23 and enables planning by predicting future representations and rewards for potential actions. We implement the world model as a Recurrent State-Space Model (RSSM) 24, shown in First, an encoder maps sensory inputs xt to stochastic representations zt . Then, a sequence model…

SLIDE 4

What they measured

We evaluate the generality of Dreamer across 8 domains—with over 150 tasks—under fixed hyperparameters. We designed the experiments to compare Dreamer to the best methods in the literature, which are often specifically designed and tuned for the benchmark at hand. We further compare to a high-quality implementation of PPO5 , a standard reinforcement learning algorithm that is known for its robustness. We run PPO with fixed hyperparameters chosen to maximize performance across domains and that reproduce…

SLIDE 5

Where it breaks

Not recoverable from the parsed text. Read this section in the paper yourself.

SLIDE 6

One-line takeaway

Unified training recipe across many domains with fixed hyperparameters

Assembled from the paper's own PDF, parsed with its layout intact, 102,522 characters of it, then split on the paper's own section headings. Extractive, not generated: every sentence here is lifted from the paper. The takeaway line is the club's own one-liner from its reading list. Check it before you present it.

Figures worth putting on a slide

Figure 1: Benchmark summary. a, Using fixed hyperparameters across all domains, Dreamer outperforms tuned expert algorithms across a wide range of benchmarks and data budgets. Drea
Figure 2: Diverse visual domains used in the experiments. Dreamer succeeds across these domains, ranging from robot locomotion and manipulation tasks over Atari games, procedurally
Figure 3: Training process of Dreamer. The world model encodes sensory inputs into discrete representations zt that are predicted by a sequence model with recurrent stat
Figure 3.](#page-2-0) First, an encoder maps sensory inputs xt to stochastic representations zt . Then, a sequence model with recurrent state ht pr
Figure 4: Multi-step video predictions of a DMLab maze (top) and a quadrupedal robot (bottom). Given 5 context images and the full action sequence, the model predicts 45 frames int
Figure 5: Fraction of trained agents that discover each of the three latest items in the Minecraft Diamond task. Although previous algorithms progress up to the iron pickaxe, Dream

Presented at

Saturday, May 9, 2026

Saturday Robotics, session 7

Robotics & World Models Reading Club 07: Learning to Dream: World Models, Imagination, Path to Foundation Models for Control — Los Altos

Listed on the event page as “Mastering Diverse Domains through World Models (2023)”. The arXiv title above is the record.

Why the club picked it. Unified training recipe across many domains with fixed hyperparameters