Philipp Schmid • 1/30/2025

Mini-R1: Reproduce Deepseek R1 „aha moment“ a RL tutorial

This technical tutorial details how to replicate the reinforcement learning 'aha moment' from the DeepSeek R1 paper, where the model learned self-verification. It guides readers through using Group Relative Policy Optimization (GRPO) and Q-LoRA to train an open model on the Countdown numbers puzzle, covering setup, sample generation, and distributed training with Deepspeed and vLLM.

0 comments

#Reasoning #Reinforcement Learning #Grpo