Learning to drive from images
In CarRacing, the agent receives a picture of the track and controls steering, acceleration and braking. I used this simulator to experiment with DreamerV3, which learns a model of its environment and trains a policy on trajectories imagined through that model.
The algorithm is by Hafner and colleagues; I used NM512’s PyTorch implementation. My contribution is the CarRacing integration, its configuration and the experimental run. This separates the underlying learning method from the work needed to give it a usable environment.
Adapting the interface between simulator and agent
I wrote a Python wrapper that resizes observations to 64 × 64, maps continuous actions and manages action repetition. It also supplies the start, termination and discount signals expected by the agent.
These conventions define the transitions used for learning: which image follows an action, where a trajectory begins and when it ends. I connected the task to the implementation’s environment selection and configuration as well. Gymnasium provides the simulator, while the wrapper translates its interface into the training system’s contract.
Keeping those transformations in an identifiable integration layer makes it possible to inspect what changed without treating the complete upstream implementation as my own work.
Running and examining the training campaign
The run totals 1,010,668 environment steps, approximately 41 hours on an RTX 4060. Average return over the final hundred episodes reaches 855, compared with roughly −37 at the start. The recorded reconstruction loss falls from 437 to 1.62.
These measurements describe progress within this experiment. The video shows the best evaluated episode, which completes the circuit in 339 steps. It displays the low-resolution observations available to the agent, making the information behind its driving behaviour visible.
Defining what the result establishes
The wrapper reapplies the same random seed on every reset, so training and evaluation use a single circuit. The score measures learning on that track rather than a CarRacing benchmark result on unseen circuits.
I preserved episode metrics, curves and the integration code so the campaign can be examined. The project brought together environment adaptation, a substantial training run and interpretation of its measurements. Extending the protocol to several circuits is the next experimental step.