Can a machine learn without being given the rules?

That time, that place

For a long time artificial intelligence meant writing down rules. To recognise a cat you had to define the features — pointed ears, whiskers, and so on.

It did not work. The world's cats were too various.

Neural networks were an alternative, and they had failed twice, in the 1970s and the 1990s. The field called those periods winters.

Why this question

Hinton was among the few who kept working on them through the winters.

The idea is simple: don't define the features, show it many examples and let it find them, correcting a little each time it is wrong, millions of times.

It had failed for three reasons: not enough examples, computation too slow, and deep stacks of layers refusing to train. By around 2010 the first two had dissolved. The internet had piled up images and the graphics chip had taken over the arithmetic.

What was found

In the 2012 image recognition contest, a network from Hinton's lab won by a wide margin. The error rate fell from 26 percent to 16.

From the next year almost every entrant switched to the approach. Translation, speech recognition, photo search and conversational AI all descend from it.

In 2023 Hinton left Google and began speaking publicly about the risks — of the thing he had pushed for his whole life. He received the Nobel Prize in Physics in 2024.

The man who sat through two winters is issuing warnings now that spring has come.