Models, parameters and training dynamics

A model is still a function that turns input into output. At Intermediate level, the useful question is what kind of function it is and what it can store. Modern models are flexible pattern machines. They are not a library of facts. They are a set of learned behaviours shaped by data and training.

The behaviours live in parameters. You can think of them as tiny knobs inside the model. Training turns knobs until the model produces outputs that match the examples often enough.

More parameters gives the model more capacity. It can represent more subtle patterns. It can also memorise noise if you let it.

Scale matters because the world is messy. If you want a model to handle more languages, more topics, or more edge cases, it usually needs more capacity and more data. Scale is not a free win. It increases cost, increases training time, and increases the number of ways training can go wrong.

Training is a loop where you show examples, let the model predict, measure how wrong it was, and then update the parameters so it is a little less wrong next time. That measure of wrongness is called the loss.

Loss is not a moral judgement. It is a learning signal.

Under the hood, the model uses a gradient to decide how to adjust parameters.

You do not need the equations yet. The intuition is enough. If the loss is high, the gradient points toward changes that should lower it. If the training setup is stable, repeating this process steadily improves performance.

This is where overfitting shows up.

Underfitting is the opposite. The model is too simple or not trained enough, so it cannot learn the pattern even on the training data.

The goal is generalisation.

Training is expensive and brittle because small changes can have large effects. Data quality issues, hidden leakage, unstable learning rates, poor shuffling, or a mismatch between training and real inputs can collapse learning. Even when training works, the model can learn shortcuts. It might latch onto the background of an image instead of the object. It might learn the formatting of an email instead of the message. These failures are not rare. They are the default unless you design against them.

When someone tells me “the model is accurate”, I do not argue. I ask questions. Accurate on what, measured how, and compared to which baseline. Then I ask the question people avoid: what is the model using to be accurate. Shortcut learning is the usual culprit. The model found a pattern that is true in the training data and fragile in reality.

A practical example. Imagine you are classifying “urgent” support tickets. If most urgent tickets were written by the night shift for three months, a model may quietly learn that “night shift writing style” means urgent. It looks brilliant in testing and then falls apart when staffing changes. This is why I care about datasets like I care about code. They contain assumptions, and assumptions are where failures breed.