Distillation overview
Distillation trains a student LLM using supervision from a teacher. You can transfer a large model's behavior into a smaller model, combine specialist models into one agent, or teach a model to solve tasks without hints it needed during training. The teacher supplies the learning signal; the student is the model you evaluate and deploy.
Start with the teacher–student relationship, then work through the chapters below. The examples build on SFT and token-level RL updates.
Read from basics to advanced techniques
| Chapter | What you will build |
|---|---|
| Distill a larger model | A small student trained with a stronger teacher's sampled reverse-KL feedback. |
| On-policy self-distillation | A student that learns from a solution-informed version of itself. |
| Self-distillation with River OPSD | Aligned student and teacher streams for River's native full-vocabulary forward-KL loss. |
| Multi-teacher distillation | One student learning from domain specialists through explicit task routing. |
Choose a distillation recipe
Two decisions are independent: who generates the response, and how you compare the teacher and student. On-policy distillation means the student generates the responses used for training. It does not prescribe a KL direction. This separation is central to Generalized Knowledge Distillation (GKD).
| Recipe | Who generates? | What the teacher provides | When to use it |
|---|---|---|---|
| Distillation through SFT | Teacher | Demonstrations | You have teacher text, including from a different tokenizer. |
| On-policy distillation (OPD) | Student | Probabilities at the student's response prefixes | A stronger teacher can score the student's own attempts. |
| On-policy self-distillation | Student | Predictions conditioned on a solution or feedback | Extra training context makes the same model a better teacher. |
| Multi-teacher OPD | Student | Feedback from a specialist chosen for each task | You want one model to acquire several teachers' capabilities. |
Teacher-generated SFT remains a useful baseline. DeepSeek-R1's smaller distilled models were fine-tuned on curated R1-generated examples; that experiment did not require aligning R1's vocabulary with Qwen's or Llama's. DeepSeek-R1, §2.4
What current research supports
The following is a reading of public methods as of September 25, 2026, not a ranking of models. Large-scale reports support specialist-to-student OPD; they do not establish one universally best estimator or teacher schedule.
| Source | Relevant result or design | What to take into your experiment |
|---|---|---|
| MiMo-V2-Flash, §4.4 | Domain teachers provide a teacher–student log-probability difference at each sampled token, optionally combined with outcome advantages. | Start with task routing and dense token feedback; add verified task rewards separately. |
| MOPD | Studies specialist consolidation with sampled-token and corrected top-k objectives. | Treat specialist selection and the estimator as separate experimental choices. |
| Moonshot's Kimi K3, §4.1.3 | Routes by domain and reasoning effort across nine teachers; clips sampled log-ratio rewards. Reports no clear benefit from top-k in that setting. | Sampled-token feedback is a serious baseline, including for agents. |
| DeepSeek-V4.1-Flash, §5.2.4 | Uses full-vocabulary OPD with more than 40 teachers and adjusts data and teacher mixtures during training. | Full distributions are another demonstrated choice, with substantial systems requirements. |
| SDFT and SDPO | Use demonstrations or environmental feedback to create a better-informed self-teacher. | Test whether extra context actually improves the teacher before distilling it. |
The basic walkthrough uses sampled reverse-KL feedback. It is small enough to inspect and works with River's core RL interface. It is not a reproduction of any lab's full training system. Self-distillation is a newer family with different divergences, teacher updates, and filtering rules; the acronym alone does not specify an algorithm.