← Back to Neuron LabSource on GitHub

How Neuron Lab works

Neuron Lab is a small, visual neural network you can poke at. This page explains what you are looking at, from one neuron up to a network that learns.

Open the app

A neuron is a tiny decision maker

An artificial neuron takes some numbers in and gives one number out. In between it does three things.

x₁ x₂ Σ out × w₁ × w₂ + bias activation
  1. Weigh. Multiply each input by its weight. A big weight means that input matters a lot. A negative weight argues against firing.
  2. Sum. Add the results together with a bias, a fixed nudge that makes the neuron more or less eager.
  3. Squash. Pass the sum through an activation function to get the output.
z = x₁·w₁ + x₂·w₂ + bias out = activation(z)

The first tab of the app shows exactly this, with the real arithmetic printed next to the diagram. Line thickness is the size of a weight, blue is positive and orange is negative.

Activation functions

Without an activation function a network could only ever do multiplication and addition, no matter how big it was. The activation adds the bend that lets it model interesting things.

Step

Off below zero, on at zero or above. Easy to understand, but it cannot be trained gradually.

Sigmoid

A smooth switch from 0 to 1. Good for outputs that act like a confidence.

Tanh

Like sigmoid but centered on zero, running from −1 to 1.

ReLU

Passes positive numbers and turns negatives into 0. Simple and common in modern networks.

From one neuron to a network

A single neuron can only split the world with one straight line. That is enough to separate two clumps of points, or to behave like an AND or OR gate. It is not enough for XOR, where the two classes sit on opposite diagonals, or for a ring inside a ring.

The fix is to stack neurons in layers. Layers between the input and the output are called hidden. Each hidden neuron draws its own line, and the output neuron combines those lines into a curved or multi-part shape. In the network lab each small square shows what one neuron responds to across the whole board, so you can watch the pieces being combined.

How the network learns

Nobody sets the weights by hand. They start random, and training improves them in a loop.

Each press of Train repeats this loop many times a second. The loss chart falling is the network getting better, and the dark-ringed dots disappearing are the mistakes being fixed. The Learning speed setting is the step size: too slow and nothing seems to happen, too fast and the network can overshoot.

Things to try

Under the hood

Neuron Lab is plain HTML, CSS and JavaScript with no libraries and no build step. The network, the Adam optimizer and the backpropagation are written out in the page's own script, so you can read them. Training happens entirely in your browser, and nothing is sent anywhere.

Real networks follow the same idea at a vastly bigger scale: billions of weights instead of a few dozen, and many layers, but still weigh, sum, squash, score and nudge.

← Back to Neuron LabBack to top