How a neural network learns: a playground

On October 10, 2026 By @thinquanautIn Neuro meets AI

The figure below is a complete neural network: two inputs, a handful of hidden units you choose, one output, trained by gradient descent on whatever you point it at, running in your browser. It is the smallest network that shows the things that matter about large ones, and the point of this page is to let you see them happen rather than read about them. Each experiment is a couple of clicks. If the words neural network, hidden unit or training are new, read the beginner’s guide first; if you have done lesson 3, you have already built the version of this with no hidden units.

Loading the interactive figure…

The playground. Pick a dataset, choose how many hidden units, press train. Filled dots are training examples, hollow dots are the held-out test set, and the shading is the network’s current answer for every point in the plane.

What you are looking at

Each dot is an example with two measurements (its position) and a label (its colour). The network's job is to learn a rule that gives the right colour from the position. The big square shows the rule it currently has: for every point on the plane, blue or orange, stronger where it is more sure. The small diagram on the right is the network itself, with each weight drawn as a line whose thickness is its size and whose colour is its sign. The curves below it are the loss, a measure of how wrong the network is, on the training examples (solid) and on the examples it never sees during training (dashed).

Training is the same loop as lesson 3: show the network the training examples, measure how wrong it is, work out for every weight which direction would make it less wrong, and move each weight a little in that direction. The learning rate is how little. One press of "one step" does that once; "train" does it a few times per frame until you stop it.

Five experiments

A straight line is enough. Choose two blobs and set hidden units to none. Press train. This is logistic regression, exactly lesson 3's decoder, and it can only draw a straight boundary. For two blobs a straight line is all you need, and both accuracies go to about 95 percent within a hundred epochs. Now watch the weights in the diagram: two lines, one from each input, and their thickness is the whole model.
Rings need a bend. Choose rings, still with no hidden units, and train. It cannot do it: there is no straight line that separates an inner disc from an outer ring, and the accuracy stays near 50 percent however long you wait. Set hidden units to 1 and reset: a little better, since one unit can bend the boundary once. Set it to 3: the boundary closes into a loop and the accuracy climbs past 95 percent. Each hidden unit draws one straight line of its own; the output combines them, and three lines can enclose a region.
The problem that stalled the field. Choose four corners: blue where both inputs have the same sign, orange where they differ. With no hidden units this is impossible, and in 1969 Minsky and Papert used exactly this example to argue that single-layer networks were a dead end, which helped stop research for a decade. Two hidden units solve it. Watch how: each learns one diagonal, and the output learns to say "one but not both".
Overfitting, live. Choose spiral, set 8 hidden units and a learning rate of about 1.5, and let it train for a while. The training loss keeps falling. The dashed test loss falls, flattens, and then, often, begins to rise, while the shading grows strange tongues that fit single training dots. The network is memorising the examples rather than learning the rule, and the only thing that reveals it is the data you held out. This is why lesson 3 insisted on a test set, and why every serious model is judged on data it has not seen.
Real neurons. Choose real: two neurons. The dots are the 340 trials from lesson 3, positioned by how many spikes neurons 141 and 190 fired, coloured by whether a stimulus was on the screen. With no hidden units the network reaches about 97 percent, which is lesson 3's result for these two cells. Now add hidden units. Nothing improves, because the two classes overlap where they overlap and no boundary, however bendy, can separate what the neurons themselves did not. More model does not fix less information. Lesson 3 got to 99.7 percent not with a cleverer network but with 64 more neurons.

Things to notice along the way

The learning rate is a dial between slow and unstable. Set it very low and training crawls. Set it to 3 on the spiral and the loss jumps around instead of falling: each step overshoots. Real training uses schedules that start high and shrink, for this reason.
Reset changes the answer. "Reset weights" starts from different random values. On the spiral, train twice from different starts and you will get two different boundaries with similar accuracy. There is no single right network, only a family of acceptable ones, which is one reason the weights of a trained model are hard to interpret.
Width buys shape, not understanding. Hidden units are straight lines; more of them means a boundary with more bends. That is the entire source of a network's flexibility, in two dimensions and, with more layers, in billions.
Nothing here knows what a ring is. The network has no concept of circles, corners or spirals. It has weights and a gradient. Everything that looks like understanding in the shading is the gradient having found a boundary that lowers the loss on these dots. Keep that in mind when a much larger version of the same machine produces a paragraph.

How it relates to the rest of the site

The network with no hidden units is the logistic regression of lesson 3, and the GLM primer explains why that is one member of a family. The hidden layer is new, and it is what turns a linear decoder into a universal one: a theorem from 1989 says that one hidden layer with enough units can approximate any reasonable boundary, which the rings and spiral experiments demonstrate in miniature. The train/test split is lesson 3's central rule, and the overfitting experiment is the reason for it. The real-data experiment is a reminder that in neuroscience the data, not the model, are usually the limit, which is the recurring lesson of lesson 4 and of the whole Foundations track.

The network is written in about a hundred lines of plain JavaScript inside this site's plugin: a forward pass, a backward pass computed by hand, and a loop. There is nothing in it that lesson 3 did not already contain except the hidden layer and the chain rule. The real dataset is neurons 141 and 190 from the Steinmetz recordings (Nature 2019, CC-BY, via Neuromatch).

How a brain works, how a language model works, and where they meet

On October 10, 2026October 10, 2026 By @thinquanautIn Neuro meets AI

This page is for someone who has heard that artificial intelligence was inspired by the brain, that language models are “neural networks”, and that nobody quite agrees on what either machine is doing. It explains, with no equations and no code, what a neuron does, what a brain does with billions of them, what a language model does, and where the two really overlap. The pictures move. Each one is a cartoon of the mechanism, drawn to be true in spirit rather than in detail, and the text says where the cartoon ends.

One neuron, one unit

Start with the smallest parts. A neuron is a cell. It receives signals from other neurons on its branches (dendrites), it has an electrical voltage across its skin, and when that voltage is pushed high enough it fires: a spike, a thousandth of a second long, that travels down its axon to the thousands of neurons it connects to. Between spikes the voltage leaks back to rest, so inputs that arrive close together count for more than inputs spread out in time. A neuron is a device for deciding, moment by moment, whether enough has happened recently.

An artificial unit, the building block of a language model, was invented in the 1940s and 50s as a cartoon of exactly that. It receives numbers. Each number is multiplied by a weight, which stands in for how strong a connection is. The products are added up and the sum is squashed into a fixed range, and that is the output. There is no voltage, no leak, no spike and no time: the unit is a formula, evaluated once.

Loading the interactive figure…

Watch them side by side. Left: spikes arrive, the voltage fills and leaks, and the neuron fires when it reaches threshold. Right: numbers arrive, meet their weights, are summed and squashed, and a number comes out.

Both are summarised by one idea: add up your inputs, weighted by how much you trust each source, and respond when the total is large. That idea is the whole inheritance from neuroscience to artificial intelligence, and lesson 1 builds the neuron version in thirty lines of Python so you can see the leak and the threshold do their work.

A brain: billions of them, in time

A brain is about 86 billion neurons wired into one another, each receiving from thousands and sending to thousands. Three things about how they work together are easy to miss from the single-neuron picture.

It is sparse. At any instant only a small fraction of neurons are firing; the typical neuron fires a few times a second, many fire less than once. Most of the brain is nearly silent most of the time, and that silence is informative: which cells are active, out of the many that could be, carries the message.
It runs in time. A spike is an event with a precise moment. Information is carried by when cells fire relative to each other, not just whether they fire, and rhythms (the alpha and theta waves of lesson 6) organise the timing across whole regions.
It is organised by what it does. Different regions handle vision, hearing, movement, memory and planning, each with its own wiring, connected in loops rather than in a single direction. There is no input end and output end; the system is always in the middle of doing something.

A language model: the next word, a trillion times

A large language model is a program with one job: given the words so far, produce a probability for every word that could come next. Type "The mouse saw the", and it reports that "cheese" is likely, "cat" fairly likely, "screen" possible, and "democracy" not. Pick a word from that list, add it to the text, ask again, and you have a machine that writes. Everything a chatbot does, answering, summarising, coding, is this loop.

Inside, the words are converted to lists of numbers, and those numbers pass through dozens of layers of the artificial units from the first figure, hundreds of billions of weights in all. One ingredient makes modern models different from earlier ones: attention. At each layer, each word position works out how much to draw on every earlier position. Reading "the" at the end of "The mouse saw the", the model attends strongly to "mouse" and "saw", because those are what make "cheese" and "cat" likely. Attention is how a model keeps track of what a sentence is about, and it is learned, not programmed.

Loading the interactive figure…

Left: a cartoon of a patch of cortex, with spikes rippling through a sparse network a few cells at a time. Right: a cartoon of a language model reading a sentence, attending to the earlier words, and producing probabilities for the next one. The sentences and numbers are illustrative.

How does it learn the weights? By reading. The model is shown trillions of words of text, one at a time, asked to predict the next, and told how wrong it was. A procedure called backpropagation works out, for every one of the billions of weights, which direction would have made the prediction less wrong, and nudges each by a tiny amount. Repeat for months on thousands of computers. No one writes the rules of grammar or the facts in; they are whatever settles into the weights because they make the next word more predictable.

Where they differ

	A brain	A large language model
Basic unit	A neuron: a cell that fires brief electrical spikes	An artificial unit: a weighted sum of numbers, squashed
How many	About 86 billion neurons, each with thousands of connections	Hundreds of billions to trillions of weights, arranged in layers
Signal	Spikes: all-or-nothing events, timed to the millisecond	Numbers, passed through the layers once per word
Activity	Sparse: most neurons are nearly silent most of the time	Dense: every unit computes on every pass
Time	Built in: everything is a process unfolding at its own pace	Not built in: a sequence is handled as a list of positions
Learning	Synapses strengthen or weaken from local activity; nobody has found a global error signal	Every weight is nudged by a global error signal, backpropagation, over trillions of words
Power	About 20 watts	Megawatts to train; a data centre to run at scale
What it predicts	What the senses will report next, and the outcome of actions, continuously	The next word, given the words so far

Two rows deserve emphasis. The brain learns without anything resembling backpropagation: a synapse changes based on what happens at that synapse, and no one has found the global error signal that would be needed to tell every connection how it contributed to a mistake. How brains learn as well as they do with only local rules is one of the central open questions of the field, and one of the places where neuroscience may still have something to give back to AI. And the brain runs on 20 watts, about a dim light bulb, because most of it is silent most of the time. A model that evaluates every unit on every pass cannot do that, and the gap in energy is not a detail; it is a sign that the two machines are organised on different principles.

Where they meet

Both are prediction machines. A brain is constantly predicting what its senses will report next and what its actions will bring about, and much of what it does can be read as correcting those predictions; a language model predicts the next word. Both store what they know in the strengths of connections rather than in a list of facts. Both are made of simple units that are individually dumb and collectively not. And both have turned out to be far more capable than their units would suggest, in ways their builders did not plan and do not fully understand.

Those similarities are why the two fields keep borrowing from each other: convolutional networks came from visual cortex, reinforcement learning from the dopamine system, and in the other direction, language models are now used as models of how the brain processes language, because their internal activity predicts brain recordings surprisingly well. The essay on the difference between AI and machine learning and the opening argument of this site go further into that exchange.

Words you will meet

Neuron: a brain cell that sends electrical spikes. Synapse: the junction where one neuron influences another; its strength is what learning changes.
Spike (action potential): the millisecond-long electrical pulse a neuron fires. Firing rate: spikes per second, the most common summary of what a neuron is doing.
EEG: the brain's electrical activity recorded from the scalp, which is the sum of millions of neurons at once.
Unit (artificial neuron): a weighted sum followed by a squashing function. Weight: a number standing for a connection strength; a model is its weights.
Token: the unit of text a language model reads, usually a word or part of one. Attention: the mechanism by which each position in the text draws on the others.
Backpropagation: the procedure that works out how to adjust every weight after a mistake. Training: doing that trillions of times.

Where to go from here

If this page made sense, you are ready for the learning path. Lesson 1 builds the neuron from the first figure in Python and loads a real recording of 734 of them. Lesson 3 builds the artificial unit from the same figure and uses it to read a mouse's mind. Lesson 5 shows what spikes look like coming off a real electrode, and lesson 7 turns brain rhythms into a control signal. None of it needs more than Python and an hour a week. And if you would rather see a network learn before you build one, the neural network playground trains one in your browser while you watch.

The animations on this page are cartoons of the mechanisms, built to be watched rather than measured. The neuron is a leaky integrate-and-fire model; the network is a random feed-forward sketch; the language-model sentences and probabilities are invented to show attention and next-word prediction. Real numbers for all of these are in the lessons.

Why every neuroscientist should learn to code (and why every ML engineer should learn some neuroscience)

On September 30, 2026October 1, 2026 By @thinquanautIn Neuro meets AI

Here is a fact about your brain that most machine learning engineers have never heard, and most neuroscientists have never checked for themselves with code. Load a recording of 734 neurons from a mouse brain, count how often each one fires, and you find that the typical neuron produces about two spikes per second. A third of them fire less than once a second. The network that produces thought, movement and memory is, most of the time, nearly silent.

import numpy as np

data = np.load("steinmetz_session.npz", allow_pickle=True)
spike_times = data["spike_times"]          # 734 neurons, 45 minutes, one mouse

duration = max(st.max() for st in spike_times)
rates = np.array([len(st) / duration for st in spike_times])

print(f"median firing rate: {np.median(rates):.1f} spikes per second")
print(f"neurons below 1 Hz: {np.mean(rates < 1):.0%}")
# median firing rate: 1.9 spikes per second
# neurons below 1 Hz: 33%

Eight lines. Real data, freely available, from a paper published in Nature. Compare that with an artificial neural network, where every unit in every layer is computed on every forward pass, whether it has anything to say or not. The brain runs on roughly the power of a dim light bulb. A large language model needs a building full of hardware. The difference is not a detail. It is one of the central open problems in computing, and you can see the first clue to it in eight lines of Python.

That is what this site is about.

Two tribes that should be one

Modern AI began as an attempt to copy the brain. The perceptron, the neural network, the convolutional layer inspired by the visual cortex, reinforcement learning built on the dopamine literature: the lineage is direct. Then the two fields drifted apart. Machine learning became an engineering discipline organised around benchmarks and GPUs. Neuroscience became a data science organised around ever larger recordings, without always having the people to analyse them.

The result is two groups of very smart people who rarely read each other’s papers. Most ML engineers have never looked at a spike train. Many neuroscientists still hand their data to a collaborator, or to a graduate student who learned Python last month, because writing the analysis themselves feels like someone else’s job.

I have worked on both sides of this line. I currently work on precision medicine, brain-computer interfaces and semantic search, which means I spend my days translating between the two tribes. This site is the translation guide I wish had existed when I started.

This is not a new struggle. Every generation of scientists has been limited by the cognitive workload of its own tools. Johannes Kepler spent years of hand arithmetic wrestling the orbit of Mars into an ellipse, and later called it his war with Mars. For most of the last two centuries, the word computer meant a person: rooms of people, very often women, grinding through calculations for astronomers, engineers and, eventually, the first neuroscientists.

The clearest example in our own field is from 1952. Alan Hodgkin and Andrew Huxley had worked out the equations that describe how a nerve impulse travels along an axon, work that later won them a Nobel Prize. The Cambridge computer was out of service, so Huxley solved the equations by hand on a mechanical desk calculator. Computing a single propagated action potential took him about three weeks. On the laptop I am writing this on, the same calculation takes a few milliseconds.

The lesson is not that we are smarter than Huxley. It is that the bottleneck of science has always been computation, and the people who could do the computation decided what got discovered. Learning to program is how you stop being on the wrong side of that bottleneck.

Why every neuroscientist should learn to code

The data outgrew the spreadsheet a decade ago

A single Neuropixels probe records from hundreds of neurons at thirty thousand samples per second. A modern imaging session produces terabytes. An EEG study has dozens of channels, hundreds of trials, tens of participants. You cannot look at this data. You can only compute on it. The person who can write the computation controls what questions get asked.

Reproducibility is a property of code, not of intentions

A result that lives in a chain of clicks in a GUI cannot be repeated exactly, not even by you, six months later. A result that lives in a script can be re-run, reviewed, shared, and corrected. The reproducibility crisis in the life sciences has many causes, but one of the fixable ones is that analysis is still often done by hand.

The models are code

Every serious theory of how neurons compute is now expressed as a simulation. Hodgkin and Huxley wrote their model as differential equations in 1952 and solved them by hand-cranked calculator. Today the same model is thirty lines of Python. If you cannot read those lines, you cannot really engage with the theory. If you can write them, you can test it against your own data.

Why every ML engineer should learn some neuroscience

The brain is an existence proof

Every time someone claims that intelligence needs a trillion parameters, or a datacentre, or a training set of the entire internet, the brain quietly disagrees. It learns language from a few million words, not a few trillion. It learns to recognise an object from a handful of examples. It does this on twenty watts. We do not know how, and that is exactly the point: the gap between what brains do and what our models do is a map of the ideas we have not had yet.

Sparsity, timing and learning without backpropagation

Those nearly silent neurons at the top of this article are sparse coding in action. Real neurons also communicate with precisely timed events, not continuous activations, which is why neuromorphic chips exist. And no one has found backpropagation in the brain. Synapses change based on information that is local to them, in time and in space. Predictive coding, Hebbian learning, and dendritic computation are all attempts to explain how a network can learn without a global error signal. Any of them could be the next big idea in machine learning. Some of them already are.

Neuroscience is where the interesting problems are

Decoding intention from motor cortex to drive a prosthetic arm. Predicting a seizure minutes before it happens. Restoring speech from neural activity. Finding which patients will respond to which treatment from a brain scan. These are machine learning problems with unusual data, brutal constraints and enormous stakes. They are also where a working engineer can still make a first-author contribution, because the field needs builders.

What this site is

A learning path, not a link collection. Every article uses real neural data and real code, and each track builds on the one before it:

Foundations. Python for people who have never programmed, taught entirely through neuroscience examples. Your first program plots a real spike train.
Signals. What EEG, spikes and fMRI actually measure, and the signal processing that turns raw voltage into something meaningful.
Data. The open datasets and file standards of the field, and how to load, clean and share neural data reproducibly.
Models. From a single leaky neuron to Hodgkin-Huxley to small spiking networks, each one built from scratch in code.
Decoding and brain-computer interfaces. Classifying intention from neural signals, ending in a minimal working BCI loop.
Neuroscience meets AI. Where the two fields overlap today: deep networks as models of cortex, predictive coding, neuromorphic hardware, and what language models do and do not tell us about the brain.
Precision medicine. Neural biomarkers, clinical machine learning, and the many ways a good model can fail in a hospital.

You do not need a neuroscience degree. You do not need to know how to program. You need Python installed, or a browser tab open to a free notebook service, and about an hour a week.

Start here

The first lesson takes you from a downloaded file to a working model of a neuron in about thirty lines of Python, using the same recording as the eight lines above. It is the most important article on this site, because after it you will know whether this path is for you.

Lesson 1: Your first neuron in Python.

Santiago Ramón y Cajal, whose drawings of neurons gave neuroscience its founding image, spent his mornings at the microscope and his afternoons at the drawing board, because the only way to truly see the structure was to render it himself. He told young scientists that anyone can become the sculptor of their own brain. That is the spirit here. The drawing board is now a text editor, and rendering the brain yourself means writing the code.

If something on this site helps you, or if you find a mistake, I would like to hear about it. You can reach me on Twitter at @thinquanaut.

Deep Blue never learned anything: the real difference between AI and machine learning

On September 30, 2026October 1, 2026 By @thinquanautIn Neuro meets AI

In May 1997 a computer beat the world chess champion. In March 2016 a computer beat one of the world’s best Go players. Both made headlines as triumphs of artificial intelligence, and both were. But the two machines had almost nothing in common. Deep Blue did not learn to play chess. AlphaGo did almost nothing but learn. The difference between those two machines is the difference between artificial intelligence and machine learning, and once you see it you will notice it everywhere.

Two words with two histories

Artificial intelligence was named in 1955, in a proposal for a summer workshop at Dartmouth College written by John McCarthy and three colleagues. Their ambition was the whole thing: machines that use language, form concepts, solve problems reserved for humans, and improve themselves. AI is the goal. It says nothing about the method.

Machine learning was named four years later by Arthur Samuel, an IBM engineer who had written a checkers program that got better by playing against itself. He defined it as giving computers the ability to learn without being explicitly programmed. Machine learning is one method. It is a way of getting to AI, or to a lot of things that nobody would call AI, by letting a program adjust itself from examples instead of following rules written by a person.

So the relationship is simple. Machine learning is a subset of artificial intelligence. Everything that learns from data to do something intelligent-looking is AI. But a great deal of AI, historically most of it, never learned anything.

flowchart TB

AI[Artificial intelligence: the goal] --> R[Rule-based systems]

AI --> ML[Machine learning: learn from examples]

R --> R1[Deep Blue, 1997]

R --> R2[Expert systems, 1970s to 80s]

R --> R3[Route planning, chess openings, tax software]

ML --> S[Supervised: labelled examples]

ML --> U[Unsupervised: find structure]

ML --> RL[Reinforcement: reward and punishment]

S --> DL[Deep learning: learned features]

DL --> D1[AlphaGo, 2016]

DL --> D2[Speech, vision, language models]

AI is the ambition. Machine learning is one family of methods for reaching it, and deep learning is one branch of that family.

The rules era

For its first thirty years, AI mostly meant writing rules. The Logic Theorist of 1956 proved theorems by searching through logical steps. Chess programs searched millions of positions per second and scored them with evaluation functions that grandmasters helped design. Expert systems of the 1970s and 80s, such as MYCIN for diagnosing blood infections and XCON for configuring computer orders at Digital Equipment Corporation, encoded the judgement of human specialists as thousands of if-then rules. XCON reportedly saved the company tens of millions of dollars a year.

This was genuinely intelligent behaviour by any reasonable standard of the time, and none of it was learned. Deep Blue is the peak of this era. It won because it could search two hundred million positions a second and because its evaluation function had been tuned by hand over years. Show it a new game and it would have been helpless.

The rules era ran into a wall that its own practitioners named: the knowledge acquisition bottleneck. Every rule had to be extracted from a human expert and written down. Experts disagreed, could not articulate what they knew, and the rules interacted in ways nobody could predict. The systems were brittle, expensive to build, and impossible to maintain. By the late 1980s funding collapsed in what is now called the second AI winter.

The learning era

The alternative had been there since the beginning. Frank Rosenblatt’s perceptron of 1958 learned to classify simple images by adjusting weights from examples, and Samuel’s checkers program learned from self-play. But learning needed two things the rules approach did not: large amounts of data and large amounts of computation. Neither was available.

Three things changed. The backpropagation algorithm, popularised in 1986, made it practical to train networks with many layers. The internet produced data at a scale nobody had imagined. And graphics cards built for video games turned out to be ideal for the arithmetic that neural networks need. The moment usually cited as the turning point is 2012, when a deep network called AlexNet cut the error rate on the ImageNet image recognition benchmark by a margin that made the entire field switch approaches within about two years.

AlphaGo is the peak of this era so far. Go has too many positions to search the way Deep Blue searched chess, and no human could write an evaluation function for it. So AlphaGo learned one, first from thirty million positions from expert games, then from millions of games against itself. Its successor, AlphaZero, dropped the human games entirely and learned chess, Go and shogi from nothing but the rules, and beat the best rule-based chess engine in the world in a matter of hours.

Spot the difference in daily life

The distinction is not academic. It tells you how a system will fail, what it needs to work, and who is responsible when it is wrong.

Task	Rule-based approach	Learned approach
Filtering spam	Block messages containing certain words. Beaten within weeks by misspellings.	Since about 2002, classifiers trained on millions of labelled messages. Adapts as spam changes.
Navigation	Shortest-path search, as in every GPS. No learning needed; the road graph is known.	Arrival-time prediction, which is learned from traffic history.
Filing taxes	The tax code is rules. Tax software encodes them exactly. A learned system here would be a liability.	Fraud detection on the returns, learned from past cases.
Reading handwriting	Attempts to describe letter shapes as rules failed for decades.	Networks trained on handwritten digits have read postal codes and bank cheques since the early 1990s.
Detecting spikes in a neural recording	A voltage threshold, chosen by the experimenter. Still the first step in most pipelines.	Sorting the detected spikes into neurons, which is clustering, a form of unsupervised learning.

Notice that the rule-based column is not the losing side. Your GPS, your tax software and your compiler are all rules, and they should be. The learned column wins only where the rules cannot be written down, which is the subject of the next article.

Where neuroscience sits in this

The brain was the inspiration for the learning side from the start. Rosenblatt was a psychologist, and the perceptron was explicitly a model of how neurons might learn. The modern neural network keeps the name and the rough idea, a network of simple units whose connections strengthen with experience, and discards nearly everything else about biology. That gap is the subject of the Neuroscience meets AI track on this site: what the brain does that our learners do not, and which of those differences matter.

It also cuts the other way. Neuroscience today uses both columns of the table above. A threshold detector is a rule. A spike sorter is a learner. A decoder that turns motor cortex activity into cursor movement is a learner, but the safety limits around it are rules. Knowing which is which is part of being able to build and trust these systems.

Further reading: Nils Nilsson, The Quest for Artificial Intelligence, is the standard history and is free online. Arthur Samuel’s 1959 paper on checkers is short and still readable.

Could you write the rules down? A field guide to when a problem needs machine learning

On September 30, 2026October 1, 2026 By @thinquanautIn Neuro meets AI

In 1954 a team from IBM and Georgetown University demonstrated a computer translating Russian sentences into English. It handled about sixty sentences using six grammar rules and a vocabulary of 250 words. The press reported that machine translation would be solved within five years. Twelve years later a government committee, the ALPAC report of 1966, concluded that after all that funding, machine translation was slower, worse and more expensive than a human, and recommended the money stop. It did, for two decades.

The problem was not effort or talent. It was that the researchers had picked the wrong kind of solution for the problem. Translation looked like a rules problem. It is not one. Knowing the difference is one of the most useful judgements a programmer can make, and it is the subject of this article.

The one question

Before anything else, ask: could a competent person write down the rules? Not in principle, in practice, on paper, completely enough that a program following them would be correct.

If yes, write the program. Sorting a list, computing payroll, validating a form, routing a packet, rendering a web page: these are rules problems, and every one of them has a correct, testable, explainable solution that a person wrote. Using machine learning here is not just unnecessary. It replaces a correct program with an approximate one, and hides the logic where nobody can inspect it.

If no, ask why not. There are three common reasons, and only some of them point to machine learning.

flowchart TD

Q1{Can a person write the rules down completely?} -- yes --> P[Write a regular program]

Q1 -- no --> Q2{Why not?}

Q2 -- The answer is a search or optimisation --> S[Use an algorithm: search, dynamic programming, solvers]

Q2 -- Humans do it but cannot say how --> Q3{Do you have many examples with the right answers?}

Q2 -- The pattern keeps changing --> Q3

Q3 -- no --> G[Get data first, or use rules as a stopgap]

Q3 -- yes --> Q4{Is an occasional wrong answer acceptable?}

Q4 -- no --> H[Rules or human review around any model]

Q4 -- yes --> ML[Machine learning is the right tool]

The decision in one diagram. Most problems exit at the first or second box.

Reason 1: the rules exist, but the answer is a search

Finding the fastest route across a city, scheduling flights, packing a truck, playing chess. Nobody can write down the answer, but the rules of the problem are perfectly clear. These are algorithm problems, and computer science has spent seventy years on them: shortest-path search, dynamic programming, constraint solvers, linear programming. They give exact or provably near-optimal answers, and they do not need training data. Reaching for machine learning here is a common and expensive mistake.

Reason 2: humans do it easily but cannot say how

Recognising a face. Understanding a spoken sentence. Telling a cat from a dog. Reading a handwritten postal code. Deciding whether an email is spam. You do these effortlessly, and if asked to write down the rules you would fail, as decades of researchers did before you. This is the home territory of machine learning: the knowledge exists, in human judgement, and can be captured as examples even though it cannot be captured as rules.

Translation is here. The rules approach failed for forty years. In the early 1990s an IBM group tried something that sounded like giving up: instead of grammar, feed the computer millions of sentence pairs from the Canadian parliament, which publishes everything in English and French, and let it learn which words and phrases tend to correspond. It worked better than the rule systems almost immediately. Frederick Jelinek, who led IBM’s speech group, is remembered for the remark that every time he fired a linguist, the recogniser improved. Modern translation is the descendant of that decision.

Reason 3: the rules keep changing

Spam filters are the clean example. In the late 1990s, filters were lists of forbidden words. Spammers responded with misspellings and images within weeks, and every update to the rules was answered by an update to the spam. In 2002 Paul Graham published a short essay proposing a filter that learned from each user’s own mail, and within a few years every major email provider used a learned filter. The point was not that the learned filter was cleverer. It was that it could be retrained overnight, while the rule-writers could not keep up. Fraud detection, recommendation, and ad targeting are the same shape: the target moves, so the program has to move with it.

Two more questions before you commit

Do you have the examples?

Machine learning is a way of turning examples into a program. No examples, no program. The examples must be numerous, must include the correct answer, and must look like the cases the system will meet in use. A model trained on hospital scans from one machine can fail on scans from another. A model trained on last year’s customers can fail on this year’s. If you cannot get the data, you are not ready, and a rule-based stopgap that collects data while it runs is often the right first version.

Can you afford to be wrong sometimes?

A learned system is approximate by construction. It will be wrong on some fraction of inputs, and it will not tell you which. Where a wrong answer costs a little and can be corrected, as in a recommendation or a spam folder, that is fine. Where a wrong answer is expensive or irreversible, the model needs rules around it: hard limits, human review, or a fallback. A brain-computer interface that moves a wheelchair is a learned decoder wrapped in rule-based safety limits, and it should be.

Real problems, sorted

Problem	Verdict	Why
Calculate a mortgage payment	Program	The formula is the rule.
Find the fastest driving route	Algorithm	Rules are known; the answer is a search.
Predict how long the route will take at 5 pm	Machine learning	Depends on patterns in traffic history nobody can write down.
Detect that a credit card is being used fraudulently	Machine learning, with rules	The pattern changes constantly; a few hard rules catch the obvious cases.
Read the amount on a handwritten cheque	Machine learning	Humans can do it and cannot explain how. Solved by neural networks in the 1990s.
Compile a program	Program	Every rule is specified. Exactness is the whole point.
Decide which neuron produced each spike in a recording	Machine learning (clustering)	No rules exist; the spike shapes have to be grouped from the data.
Flag a seizure in an EEG recording	Machine learning, with rules	The signature is learnable; the alert logic around it should be explicit.
Schedule operating rooms in a hospital	Algorithm	A constraint problem with clear rules, not a pattern to learn.

A test you can run on your own problem

Try to write the program. Spend an hour, or a day, writing the rules as if you had to. One of three things happens:

You finish, and you have a program. It is probably better than any model would have been.
You realise the rules are clear but the answer is a search or an optimisation. Look for the algorithm before you look for the model.
You keep adding exceptions, the exceptions need exceptions, and you can feel the rules turning to mush. That feeling is the signal. If you have examples, and being wrong sometimes is acceptable, this is a machine learning problem.

The researchers of 1954 never ran this test, because it was not yet possible to fail it honestly. You can. It takes an afternoon and it can save a year.

Further reading: the ALPAC report of 1966 is short and surprisingly readable. Paul Graham’s essay A Plan for Spam (2002) is the best single account of why learning beat rules for a moving target.