This page is for someone who has heard that artificial intelligence was inspired by the brain, that language models are “neural networks”, and that nobody quite agrees on what either machine is doing. It explains, with no equations and no code, what a neuron does, what a brain does with billions of them, what a language model does, and where the two really overlap. The pictures move. Each one is a cartoon of the mechanism, drawn to be true in spirit rather than in detail, and the text says where the cartoon ends.
One neuron, one unit
Start with the smallest parts. A neuron is a cell. It receives signals from other neurons on its branches (dendrites), it has an electrical voltage across its skin, and when that voltage is pushed high enough it fires: a spike, a thousandth of a second long, that travels down its axon to the thousands of neurons it connects to. Between spikes the voltage leaks back to rest, so inputs that arrive close together count for more than inputs spread out in time. A neuron is a device for deciding, moment by moment, whether enough has happened recently.
An artificial unit, the building block of a language model, was invented in the 1940s and 50s as a cartoon of exactly that. It receives numbers. Each number is multiplied by a weight, which stands in for how strong a connection is. The products are added up and the sum is squashed into a fixed range, and that is the output. There is no voltage, no leak, no spike and no time: the unit is a formula, evaluated once.
Both are summarised by one idea: add up your inputs, weighted by how much you trust each source, and respond when the total is large. That idea is the whole inheritance from neuroscience to artificial intelligence, and lesson 1 builds the neuron version in thirty lines of Python so you can see the leak and the threshold do their work.
A brain: billions of them, in time
A brain is about 86 billion neurons wired into one another, each receiving from thousands and sending to thousands. Three things about how they work together are easy to miss from the single-neuron picture.
- It is sparse. At any instant only a small fraction of neurons are firing; the typical neuron fires a few times a second, many fire less than once. Most of the brain is nearly silent most of the time, and that silence is informative: which cells are active, out of the many that could be, carries the message.
- It runs in time. A spike is an event with a precise moment. Information is carried by when cells fire relative to each other, not just whether they fire, and rhythms (the alpha and theta waves of lesson 6) organise the timing across whole regions.
- It is organised by what it does. Different regions handle vision, hearing, movement, memory and planning, each with its own wiring, connected in loops rather than in a single direction. There is no input end and output end; the system is always in the middle of doing something.
A language model: the next word, a trillion times
A large language model is a program with one job: given the words so far, produce a probability for every word that could come next. Type "The mouse saw the", and it reports that "cheese" is likely, "cat" fairly likely, "screen" possible, and "democracy" not. Pick a word from that list, add it to the text, ask again, and you have a machine that writes. Everything a chatbot does, answering, summarising, coding, is this loop.
Inside, the words are converted to lists of numbers, and those numbers pass through dozens of layers of the artificial units from the first figure, hundreds of billions of weights in all. One ingredient makes modern models different from earlier ones: attention. At each layer, each word position works out how much to draw on every earlier position. Reading "the" at the end of "The mouse saw the", the model attends strongly to "mouse" and "saw", because those are what make "cheese" and "cat" likely. Attention is how a model keeps track of what a sentence is about, and it is learned, not programmed.
How does it learn the weights? By reading. The model is shown trillions of words of text, one at a time, asked to predict the next, and told how wrong it was. A procedure called backpropagation works out, for every one of the billions of weights, which direction would have made the prediction less wrong, and nudges each by a tiny amount. Repeat for months on thousands of computers. No one writes the rules of grammar or the facts in; they are whatever settles into the weights because they make the next word more predictable.
Where they differ
| A brain | A large language model | |
|---|---|---|
| Basic unit | A neuron: a cell that fires brief electrical spikes | An artificial unit: a weighted sum of numbers, squashed |
| How many | About 86 billion neurons, each with thousands of connections | Hundreds of billions to trillions of weights, arranged in layers |
| Signal | Spikes: all-or-nothing events, timed to the millisecond | Numbers, passed through the layers once per word |
| Activity | Sparse: most neurons are nearly silent most of the time | Dense: every unit computes on every pass |
| Time | Built in: everything is a process unfolding at its own pace | Not built in: a sequence is handled as a list of positions |
| Learning | Synapses strengthen or weaken from local activity; nobody has found a global error signal | Every weight is nudged by a global error signal, backpropagation, over trillions of words |
| Power | About 20 watts | Megawatts to train; a data centre to run at scale |
| What it predicts | What the senses will report next, and the outcome of actions, continuously | The next word, given the words so far |
Two rows deserve emphasis. The brain learns without anything resembling backpropagation: a synapse changes based on what happens at that synapse, and no one has found the global error signal that would be needed to tell every connection how it contributed to a mistake. How brains learn as well as they do with only local rules is one of the central open questions of the field, and one of the places where neuroscience may still have something to give back to AI. And the brain runs on 20 watts, about a dim light bulb, because most of it is silent most of the time. A model that evaluates every unit on every pass cannot do that, and the gap in energy is not a detail; it is a sign that the two machines are organised on different principles.
Where they meet
Both are prediction machines. A brain is constantly predicting what its senses will report next and what its actions will bring about, and much of what it does can be read as correcting those predictions; a language model predicts the next word. Both store what they know in the strengths of connections rather than in a list of facts. Both are made of simple units that are individually dumb and collectively not. And both have turned out to be far more capable than their units would suggest, in ways their builders did not plan and do not fully understand.
Those similarities are why the two fields keep borrowing from each other: convolutional networks came from visual cortex, reinforcement learning from the dopamine system, and in the other direction, language models are now used as models of how the brain processes language, because their internal activity predicts brain recordings surprisingly well. The essay on the difference between AI and machine learning and the opening argument of this site go further into that exchange.
Words you will meet
- Neuron: a brain cell that sends electrical spikes. Synapse: the junction where one neuron influences another; its strength is what learning changes.
- Spike (action potential): the millisecond-long electrical pulse a neuron fires. Firing rate: spikes per second, the most common summary of what a neuron is doing.
- EEG: the brain's electrical activity recorded from the scalp, which is the sum of millions of neurons at once.
- Unit (artificial neuron): a weighted sum followed by a squashing function. Weight: a number standing for a connection strength; a model is its weights.
- Token: the unit of text a language model reads, usually a word or part of one. Attention: the mechanism by which each position in the text draws on the others.
- Backpropagation: the procedure that works out how to adjust every weight after a mistake. Training: doing that trillions of times.
Where to go from here
If this page made sense, you are ready for the learning path. Lesson 1 builds the neuron from the first figure in Python and loads a real recording of 734 of them. Lesson 3 builds the artificial unit from the same figure and uses it to read a mouse's mind. Lesson 5 shows what spikes look like coming off a real electrode, and lesson 7 turns brain rhythms into a control signal. None of it needs more than Python and an hour a week. And if you would rather see a network learn before you build one, the neural network playground trains one in your browser while you watch.
The animations on this page are cartoons of the mechanisms, built to be watched rather than measured. The neuron is a leaky integrate-and-fire model; the network is a random feed-forward sketch; the language-model sentences and probabilities are invented to show attention and next-word prediction. Real numbers for all of these are in the lessons.
Keep reading
How a neural network learns: a playground
A complete neural network running in your browser: pick a dataset, choose the hidden units, press train, and watch the boundary bend,…
Why every neuroscientist should learn to code (and why every ML engineer should learn some neuroscience)
The typical neuron in a mouse brain fires twice a second and a third of them are nearly silent. You can see…
Deep Blue never learned anything: the real difference between AI and machine learning
Deep Blue beat Kasparov without learning a thing. AlphaGo did almost nothing but learn. That gap is the difference between artificial intelligence…