What is a neural network?

Picture a long line of sorters at a grain depot. The first row only checks size. The second row only checks color. The third row only checks smell. No single sorter knows what a good sack looks like, but the line as a whole does.

A neural network works like that line, and it is a stack of very simple steps. Each step takes numbers in, does a small sum, and passes numbers on. The skill lives in the whole stack, and it never lives in one step.

Nobody writes those sums by hand. Training sets them, and that is the part worth understanding.

What a neural network is made of

The basic part is called a unit, or sometimes a neuron. A unit does three things and nothing else.

1. It takes several numbers in.
2. It multiplies each one by its own weight, then adds the results together.
3. It passes that total through a simple bend, so the output does not follow a straight line.

Why the bend? Because without it, a whole stack of layers collapses into one big sum, and a sum can only draw a straight line. Therefore, the bend is what lets the network follow a curve.

Units are grouped into layers. The first layer sees the input, and the last layer gives the answer. The layers in between are called hidden, because you never look at them directly.

The word deep, and why it matters

Deep learning simply means a network with many hidden layers. However, there is no magic number that makes a network deep.

Depth buys you steps of meaning. In a network that reads photos, the early layers find edges. The middle layers find shapes such as an eye or a wheel. The late layers find whole things such as a face or a bus. Each layer therefore builds on the one below it.

This is the grain line again. Simple checks first. The verdict last.

How it learns, step by step

All the weights start random, so the first answer is nonsense. Then this loop runs.

1. Push one batch of examples through the network. This is the forward pass.
2. Compare the output with the right answer and measure the gap. The gap is called the loss.
3. Work backwards through the layers to see how much each weight added to that gap. This step is called backpropagation.
4. Move every weight a small step in the direction that shrinks the loss.
5. Repeat until the loss stops falling.

Think of finding the bottom of a valley in thick fog. You cannot see the bottom, but you can feel which way the ground slopes under your feet. So you take a small step downhill and feel again. Millions of small steps get you there. That is all training is, and it is ordinary machine learning done with a great many weights.

What sits inside a trained network

After training, the network is a file of numbers. Nothing in it says cat, or fraud, or Swahili, and there is no rule you can point at.

This is why these systems are hard to explain. You can read every weight and still not be able to say why one photo was rejected. Tools exist that show which part of an input mattered most, and they help, but they give a hint rather than a reason.

Everything the network holds came from its training data, so change the data and you change the model completely.

The main shapes you will hear about

Networks come in families, each shaped for a kind of input.

• Convolutional networks slide a small window across an image, so a thing is spotted wherever it sits in the frame. These suit photos and video.
• Recurrent networks read a sequence one item at a time and carry a memory along. These were once common for text and speech.
• Transformers let every part of the input look at every other part at once. This is the design behind a large language model.

The building block is the same in all three, and only the wiring changes.

What they are good at, and where they fail

Neural networks win when the input is rich and messy and the pattern is hard to write down. Images, sound, text and video are all like that. Given enough examples, they beat hand written rules by a wide margin.

However, they are heavy, because training costs real money and real power. They need far more examples than simpler methods. For a small table of numbers, a plain method often wins, and it is easier to defend to a regulator.

They also fail with strange confidence. Show a network something well outside its examples and it still returns an answer, with a high score attached. This is because it has no way of telling you that it has never seen anything like this before.

Where you already meet them, and what to check

Neural networks are in your phone camera, your voice notes, your bank fraud alerts, your map, your translation app and every chat assistant. They are the reason modern artificial intelligence works at all.

If someone offers you one, ask three questions. How many examples did you train on? How does it score on data it never saw? What does it do when it is unsure? A good team answers all three in a minute. THE THIRD ANSWER MATTERS MOST. A system that cannot say it is unsure will be wrong loudly, and you will be the one who has to explain it.

Just Out Tech explains new research in plain language. This article was drafted with AI assistance and checked by a human against the original source.