World Models

World Models are AI systems that learn an internal representation of environment dynamics, enabling them to predict future states given current states and actions. Rather than interacting directly with the raw environment — which can be slow, expensive, or dangerous — an agent can train and plan within the world model's learned latent space. This process is often referred to as 'dreaming,' since the agent simulates experiences rather than collecting them from reality.

A typical world model consists of three components: an encoder that maps raw observations into a compact latent representation, a dynamics model (or transition model) that predicts how the latent state evolves given actions, and a reward predictor that estimates the reward associated with latent states. Agents can then learn policies entirely within this learned simulation, generating large amounts of synthetic training data without additional environment interaction. The resulting policies are transferred to the real environment for evaluation and fine-tuning.

The concept was pioneered by Ha and Schmidhuber (2018), who demonstrated that agents trained in a world model's latent space could play racing games and other environments with high sample efficiency. Modern architectures such as Dreamer and its successors extend this approach with recurrent state-space models, enabling long-horizon prediction and planning. World models are now central to model-based reinforcement learning, video game playing, robotics, and autonomous systems.