StudyAIStudyAI
Pro
AI FundamentalsLesson 4
Lesson 410 min

How AI Learns

Weights, loss, and gradient descent — the training loop in plain English.

What you will learn
  • ✓What 'weights' are inside a model
  • ✓How a loss function measures error
  • ✓How gradient descent improves the model step by step

Explanation

Underneath, a model is a large math function with millions of adjustable numbers called weights. Learning means finding good values for those weights.

During training, the model makes a prediction, and a loss function measures how wrong it was. A bigger loss means a worse prediction.

Gradient descent then nudges every weight a tiny amount in the direction that reduces the loss. Repeat this over millions of examples and the model gradually gets better. The size of each nudge is the learning rate — too big and training is unstable, too small and it crawls.

Code Example

python
1
for batch in dataset:
2
    pred = model(batch.x)            # forward pass
3
    loss = loss_fn(pred, batch.y)    # how wrong?
4
    loss.backward()                  # compute gradients
5
    optimizer.step()                 # nudge the weights
6
    optimizer.zero_grad()            # reset for next batch
Real-world use

Training a large model can mean repeating this loop billions of times across thousands of GPUs — which is why frontier models are so expensive to train.

Common mistakes
  • • Setting the learning rate too high, which makes the loss bounce around instead of going down.
Practice

Explain the training loop (predict → measure loss → adjust) to a friend using a non-technical analogy.

Knowledge check
0/2 answered

1. What does a loss function measure?

2. Gradient descent works by...

Answer all questions to check.
PreviousNext