Google DeepMind's Dream-RSI Lets AI Agents 'Dream' Their Way to Cheaper, Faster Search

Google and DeepMind researchers have developed a new method for making AI self-improvement systems dramatically more compute-efficient. The technique, called Dream-RSI, allows agents to evaluate alternative search strategies by replaying a stored record of past attempts — a process the researchers call "dreaming" — instead of executing expensive new live runs for each strategy under consideration.

The Problem Dream-RSI Solves

Self-improving AI agents work iteratively: propose a solution, evaluate it, learn from the result, and try again — often across thousands of iterations. For complex tasks, the search space grows enormous, and deciding which approaches to pursue, run in parallel, or abandon early is critical. Existing methods are either fixed (unable to learn from experience) or adaptive (requiring many costly live runs to test alternatives). Dream-RSI takes a third path.

As an agent searches, it records its attempts and results. Rather than testing a new strategy in a live run, the agent replays it against the stored search tree — checking what would have happened under different approaches without calling the underlying model or evaluator again. Thousands of alternative strategies can be evaluated this way before the best one is applied to an actual live search. Crucially, only the search strategy changes; the underlying model generating solutions is untouched.

Benchmark Results Across Eight Tasks

The team tested Dream-RSI with Gemini 3.1 Pro and Gemini 3.7 Flash across eight tasks in three domains, comparing against baselines using identical starting conditions but fixed search strategies.

On a genomics/finance statistical computing task, Dream-RSI produced code that outpaced the sklearn and glmnet libraries on all six test datasets. With Gemini 3.1 Pro, average runtime dropped from 3,587 to 2,931 milliseconds, and the number of attempts fell from 550 to 317. A competing system called SimpleTES required 51,200 runs to reach comparable results versus Dream-RSI's 317.

On GPU kernel-writing tasks, Dream-RSI matched baseline performance while cutting the number of runs by up to 2.43x on two tasks, and achieved up to 2.09x higher performance within the same compute budget on two others, according to DeepMind.

When Explicit Instructions Backfire

In a follow-up test, researchers compared replay-based strategy testing against condensing search histories into explicit written instructions telling the agent where to search. On one GPU task, the explicit-instruction version underperformed the version without instructions. The researchers suggest overly specific guidance narrows the search space too aggressively, blocking exploration of a broader range of approaches.

Analysis also showed the learned strategy adapted its own effort dynamically — reducing attempts as performance improved, then increasing effort again when progress stalled, which coincided with further gains.

Broader Implications

Dream-RSI addresses a narrower but consequential problem in AI self-improvement: not what a model generates, but how efficiently it searches for good outputs. The approach builds on DeepMind's earlier AlphaEvolve work but operates one layer up, optimizing the search process itself rather than the solutions found within it. The finding that explicit instructions can hurt open-ended search contrasts with other guidance-based approaches and points to a genuine tension between directing an agent and constraining it.

Google DeepMind has published code and additional details on GitHub.