AlphaProof Paper

Thu 13 November 2025

I'm very excited to finally be able to share more details about how AlphaProof works! AlphaProof is the system that we used to discover the Lean proofs for the International Mathematical Olympiad 2024, reaching silver medal performance. Our full paper Olympiad-Level Formal Mathematical Reasoning with Reinforcement Learning has now been published in Nature.

: The work described was done from the beginning of 2022 to July 2024, while I was still at Google DeepMind.

The International Mathematical Olympiad

The International Mathematical Olympiad, or IMO for short, is a yearly contest in mathematics amongst the 6 ...

MuZero Intuition

Tue 22 December 2020

To celebrate the publication of our MuZero paper in Nature (full-text), I've written a high level description of the MuZero algorithm. My focus here is to give you an intuitive understanding and general overview of the algorithm; for the full details please read the paper. Please also see our official DeepMind blog post, it has great animated versions of the figures!

MuZero is a very exciting step forward - it requires no special knowledge of game rules or environment dynamics, instead learning a model of the environment for itself and using this model to plan. Even though it uses ...

MuZero talk - ICAPS 2020

Tue 27 October 2020

I gave a detailed talk about MuZero at ICAPS 2020, at the workshop "Bridging the Gap Between AI Planning and Reinforcement Learning".

In addition to giving an overview of the algorithm in general, I also went into more detail about reanalyse - the technique that allows MuZero to use the model based search to repeatedly learn more from the same episode data.

I hope you find the talk useful! I've also uploaded my slides for easy reference.

Adventures in Reinforcement Learning

Wed 11 February 2015

You may or may not have noticed that I've been working for DeepMind for a while, causing me to get exposed to lots and lots of cutting edge machine learning research. Most of that I can't share here, but there's plenty that's already public.

Reinforcement Learning

Firstly, yes, that's what all the fuzz is about. There's a great book by Sutton and Barto, Reinforcement Learning: An Introduction, with an in-progress version of the second edition available for free from their website!

The book is very good at introducing and explaining RL itself, but does not cover ...