Inside an LLM
How a raw model becomes a helpful assistant: pretraining → SFT → RLHF.
- ✓What pretraining produces
- ✓Supervised fine-tuning (SFT)
- ✓Alignment with RLHF
Explanation
An LLM is built in stages. Pretraining on a huge chunk of the internet teaches it to predict the next token — giving broad knowledge but no manners.
Supervised fine-tuning (SFT) then trains it on example conversations so it follows instructions. RLHF (Reinforcement Learning from Human Feedback) uses human preference ratings to make answers more helpful, honest, and safe.
So the 'personality' you talk to is the result of alignment layered on top of raw prediction.
The difference between a raw base model and ChatGPT is mostly SFT + RLHF — the alignment that makes it follow instructions politely.
- • Believing the model 'looks up' answers — it generates them from learned patterns.
Summarise the three training stages of an LLM in one sentence each.
1. What does pretraining teach an LLM?
2. RLHF is used to...