World Models

Train a latent dynamics model of the environment that enables a policy to learn by “imagining” future trajectories through the learned world model, reducing real-world interaction. World models learn a compressed latent representation of the environment dynamics, then train a policy entirely within this latent space. DreamerV3 encodes observations into a compact latent state, predicts future latent states and rewards using a recurrent dynamics model, and trains an actor-critic policy on latent imaginary rollouts. Because the model “dreams” trajectories, the policy can learn from far more experience than was actually collected in the real environment. This dramatically improves sample efficiency.