Models need sleep too: Google's sleep framework brings consolidation and dreaming to post-deployment learning

Deployed models face two long-standing problems: knowledge frozen at the training cutoff — what happens in the world after training never enters the parameters — and the fine-tuning usually used to teach new things tends to grind away old abilities, catastrophic forgetting. Knowledge placed in context never becomes parameter; the session ends and it is gone, and today's largest context windows still hold less than the parametric memory of a model that is in use and learning all the time. On October 27, Google Research's blog post “The sleep framework” laid out one path through these problems, with the companion paper “Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories” (arXiv 2606.03979; a version public on OpenReview since September 2025) — a framework for continued learning after deployment, named after biological sleep: the model cycles between wake and sleep phases. The wake phase processes inputs as usual, storing experience in short-term memory; the sleep phase pauses new input and completes two things offline: consolidation and dreaming. The whole mechanism runs within a fixed architecture and human-defined training rules, without changing the model's basic structure.

The official framework figure draws the cycle plainly: in the wake phase, input streams flow through an active interface into a fast short-term memory (KV cache); on entering sleep, the fragile tokens in short-term memory are first consolidated — low rank, mix of experts, specialist models, MoE processing — into consolidated parametric memory; dreaming follows consolidation, improving the model one more round with synthetic routing and reinforcement learning; at the end of a cycle, short-term memory is pruned and cleared for the next wake phase. This answers the question our memory-comparison piece left open: retrieval, write-in and distillation each trade something off, and here the “write into parameters” step is made into a sustainable cycle.

Consolidation leans on mixture-of-experts (MoE) and the Hope architecture — Google's continual-learning architecture under the Nested Learning paradigm. The approach: at the end of the wake phase, newly unlocked low-rank experts are placed into low-frequency, stable memory blocks while old parameters stay untouched — new knowledge moves into new rooms without renovating the old ones. This decides one thing: learning new knowledge no longer requires retraining the whole model; parameter growth enters as newly added experts.

How does the content of short-term memory get written into these new experts? The paper's answer is “upward distillation” (Knowledge Seeding): a smaller model distills memory to a larger one — a smaller version of the same model (fewer active parameters) as teacher, the larger version as student. It is implemented as generalized knowledge distillation (GKD): on-policy distillation, where the student learns from the teacher's distributions on self-sampled sequences; and Learning to Imitate, where the student completes prefixes of teacher text, rewarded by a blend of frozen-reward-model semantic similarity and Levenshtein-based absolute similarity. Only the newly added parameters are updated; old parameters stay frozen.

Consolidation ending is not the end of a cycle: the low-rank experts used in the sender block — the wake phase's fast short-term memory block — are reset, “a similar procedure of synaptic pruning in human brain”, freeing capacity for the next round. The blog's theory section frames consolidation as “structured, low-rank compression”: the model picks compact directions that preserve its own behavior and semantically rewarded content. For a reader, the theory says this: experience is not backed up into parameters verbatim, but compressed into a form the model itself can reuse. Consolidation saves the experience; dreaming makes the model stronger — the other half of the sleep phase.

Dreaming runs in three steps. First, random expert selection: the MoE router randomly activates non-standard experts, mixing knowledge domains to surface latent patterns — like forcing the model to think with a different set of colleagues. Second, curated rehearsal: the model generates synthetic samples (“dreams”), filtered by gradient-based data selection to keep the most instructive ones. Third, RL refinement: reinforcement-learning fine-tuning on the best dream samples, evaluated on downstream tasks. Together, dreaming is not passive consolidation but active self-improvement — the model sets its own problems and grades its own work.

The blog's theory section gives dreaming's benefit a name: it improves the “geometry of learning” — broadening explored gradient directions, yielding a more balanced optimization landscape with better-conditioned gradients. In plain words: consolidation handles remembering, dreaming handles understanding. A model that only consolidates is like a student who only takes notes — what is kept are entries; with dreaming, the model cross-practices new knowledge against other domains inside its own synthetic curriculum, turning entries into method.

Dreaming is not decoration, and the ablations say so directly: on knowledge incorporation (SQuAD n=1 / continued pretraining n=200), the full sleep framework reaches 48.9/46.2; removing the dreaming stage drops it to 35.7/36.2 — nearly back to the base model's 31.9/31.9; removing gradient-based selection gives 47.1/45.2, removing random expert 48.0/44.7. The dreaming stage contributes most of the gain from “base model” to “sleep framework”, with random experts and gradient-based selection each carrying part of it. The paper's “self-generated curriculum” is not a rhetorical flourish — it is where the effect comes from.

Math reasoning (Qwen3-8B backbone, avg@16, AIME-24/25, HMMT-25): base 73.8/68.1/42.4 → SFT 75.5/66.4/43.7 → GRPO 76.4/68.1/44.9 → OPSD 76.6/67.4/45.1 → Sleep 79.2/69.0/46.1 — AIME-24 +5.4, AIME-25 +0.9, HMMT-25 +3.7, ahead of two strong baselines. The smaller Qwen3-1.7B rises from 49.8/34.5/25.7 to 53.2/40.2/29.3, +5.7 on AIME-25, so the gains hold on smaller models too. The blog's cited 79.2/69.0/46.1 is the Qwen3-8B Sleep row.

Knowledge incorporation (SQuAD no-context, n=1 single passage / n=200 continued pretraining): base 31.9/31.9 → fine-tuned without dreaming 33.4/32.0 → SEAL 46.7/43.2 → Sleep (Transformer) 48.1/44.3 → Sleep (four-level) 48.9/46.2. Few-shot ARC: in-context learning 0%, test-time training 10%, SEAL 72.5%, Sleep 80%. The two tables share a shape: fine-tuning without dreaming adds about 1.5 points over base, SEAL (Stanford's self-edit, self-train method) already opens the gap, and the sleep framework clears it by another notch — dreaming and self-rewriting matter on the same order here.

Continual translation of unseen languages is the framework's most vivid display (lower panel of the official results figure): on Manchu→English, the no-adaptation in-context baseline collapses from 20.9 to 10.1 after the second language is learned; the sleep stages push the score from 21.1 (Sleep-1) to 22.5 (Sleep-2) and 23.9 (Sleep-3), approaching the single-language training reference. Kalamang (MTOB) goes from 35.6 to 38.1 against a no-adaptation baseline of 15.4. Class-incremental learning (Llama backbones) points the same way: on CLINC Sleep 97 leads Hope-without-sleep at 93, Banking 92 vs 84, DBpedia 89 vs 82; EWC 80/75/71 and no adaptation 78/72/70 sit lower. On BABILong the blog claims “near-perfect scaling up to 10M tokens, outperforming strong baselines like GPT-4, and Titans” — the details sit in a paper appendix we have not audited line by line, so it is cited as vendor-reported.

Limits to keep while reading: this is a research framework, not a released product model — the backbone is Google's own Hope architecture, experiments use Qwen3-1.7B/8B, Llama-3B/8B and Llama-3.2-1B base models, the math experiments train with LoRA (rank 64); every number is vendor-reported (taken one by one from the paper HTML tables and the official blog), with no reproduction on our side; the paper's main text and tables give no full training-cost accounting. The blog also states that despite the neuroscience names, this is standard machine learning — readers should not read “sleep/dreaming” as a biological mechanism reproduced. It connects back to two existing lines on the site: our October 21 memory-comparison piece (retrieval, write-in and distillation each trade off) set the then-state of the art, and the sleep framework makes “writing into parameters” a sustainable cycle; our personal-AI-companions topic's Muse chapter covers memory management, and the sleep framework is the training-side half of the answer; it shares the research mainline of the October 31 expert-following experiment piece on how deployed systems shape themselves.