How Does an AI Learn?

The age of teaching by rules

At first, rules it was. 'A cat: ears pointed, whiskers present, four legs…' Programmers stacked them up.

And it collapsed. A fold-eared cat appears. A tailless one. A photo from behind shows no whiskers. Exceptions bred exceptions, until people faced an odd truth: we recognise cats, but we cannot fully write down how.

A million little knobs

So another road opened: imitate the brain. Build millions of very simple calculating units, wire them together, and put a turnable knob on every joint. How the knobs are set decides which way signals flow.

The teaching goes like this. Show a cat photo; look at the answer. If the machine says 'dog', trace the error backwards and turn the responsible knobs, each a tiny bit, toward 'cat'. Repeat with millions of photos, millions of times.

That is the difference between programming and learning. Piece three's program was dictated line by line; learning is wrong experiences turning knobs. The same side of the street as the child and the cat.

An idea long dismissed

The idea itself is old — the 1950s. For decades it was treated as a backwater: small nets achieved little, the fashion moved on, funding dried up more than once.

A stubborn few held on, and they were proved right not because minds changed but because the materials arrived: the internet piled up mountains of photos to learn from, and piece six's gaming chip supplied the hands to turn billions of knobs at once.

In 2012 a net trained on two gaming chips upended the image-recognition contest — the very scene piece six stopped at. One of the stubborn few was on that team.

A little further in

One step to the talking machine

Today's conversational AI has the same skeleton. Only the game changed: instead of guessing cats, guess the next word. After 'once upon a time there was a' — what comes next? Play that against mountains of human writing, endlessly, and something like the grain of language grows in the knobs — no grammar taught, no knowledge inserted.

There is a price. Not being able to explain the inside means not being able to say in advance when it will be wrong. How far 'continuing plausibly' sits from 'knowing' is a question still open.

One last piece remains. None of this — the photo mountains, the billions of knobs, the training — happens inside your phone. It happens somewhere else. What that place called 'the cloud' actually is: the final piece.

The question that remainsA machine that only does what it is told has learned a division of labour no one ordered. Is that still doing what it was told?