Neuron Lab is a small, visual neural network you can poke at. This page explains what you are looking at, from one neuron up to a network that learns.
Open the appAn artificial neuron takes some numbers in and gives one number out. In between it does three things.
The first tab of the app shows exactly this, with the real arithmetic printed next to the diagram. Line thickness is the size of a weight, blue is positive and orange is negative.
Without an activation function a network could only ever do multiplication and addition, no matter how big it was. The activation adds the bend that lets it model interesting things.
Off below zero, on at zero or above. Easy to understand, but it cannot be trained gradually.
A smooth switch from 0 to 1. Good for outputs that act like a confidence.
Like sigmoid but centered on zero, running from −1 to 1.
Passes positive numbers and turns negatives into 0. Simple and common in modern networks.
A single neuron can only split the world with one straight line. That is enough to separate two clumps of points, or to behave like an AND or OR gate. It is not enough for XOR, where the two classes sit on opposite diagonals, or for a ring inside a ring.
The fix is to stack neurons in layers. Layers between the input and the output are called hidden. Each hidden neuron draws its own line, and the output neuron combines those lines into a curved or multi-part shape. In the network lab each small square shows what one neuron responds to across the whole board, so you can watch the pieces being combined.
Nobody sets the weights by hand. They start random, and training improves them in a loop.
Each press of Train repeats this loop many times a second. The loss chart falling is the network getting better, and the dark-ringed dots disappearing are the mistakes being fixed. The Learning speed setting is the step size: too slow and nothing seems to happen, too fast and the network can overshoot.
Neuron Lab is plain HTML, CSS and JavaScript with no libraries and no build step. The network, the Adam optimizer and the backpropagation are written out in the page's own script, so you can read them. Training happens entirely in your browser, and nothing is sent anywhere.
Real networks follow the same idea at a vastly bigger scale: billions of weights instead of a few dozen, and many layers, but still weigh, sum, squash, score and nudge.