Modern NLP: Small Models vs LLMs
Choosing the right tool for cost, speed, and complexity.
- ✓When a small model wins
- ✓When to use an LLM
- ✓Balancing cost, speed, and accuracy
Explanation
LLMs are powerful but slow and costly. Many NLP tasks don't need them.
For high-volume, well-defined tasks (spam, sentiment, routing), a small fine-tuned model is faster and far cheaper. For complex, open-ended tasks (summarising nuanced text, answering varied questions), an LLM shines.
A smart system often routes: a cheap model handles the easy 90%, and an LLM handles the hard cases.
A support tool might classify tickets with a tiny model and only call an LLM to draft replies for tricky ones — cutting cost dramatically.
- • Defaulting to an expensive LLM for every task, including ones a small model handles cheaply.
List two NLP tasks best done by a small model and two best done by an LLM, with reasons.
1. For a high-volume, simple task you should usually prefer...
2. When does an LLM make most sense?