The number that broke arithmetic

Elements of Mathematics

From First Principles for Data Scientists, AI Engineers and Quants

---

Knowing when an algorithm will converge, and why your computer disagrees with your mathematics

You will need from earlier: nothing. This is the first technical piece in the series, and it assumes only arithmetic.

---

Here is a question that sounds trivial and is not: how do you know a number exists?

That sounds like philosophy. It is the most practical question in this series, every optimisation algorithm, every root finder, every gradient descent routine and every neural network training loop works the same way: start from a guess, apply the same rule again and again, and produce a sequence, an ordered list of numbers, each one built from the last.

A procedure that repeatedly applies the sume rule over and over is called an iterative method. Mathematicians say such a sequence converges if its terms settle closer and closer to a single value, called its limit.

The Greeks hit this first, from an unexpected direction, and it took 2,200 years to answer properly.

The research programme that killed itself

The Pythagoreans held a position that sounds naive until you try to refute it: every magnitude in the world is a ratio of whole numbers, one quantity measured against another, like $3$ parts to $2$. Lengths, areas, musical intervals, all of it. The world was discrete underneath and the continuous appearance was bookkeeping.

This was not mysticism dressed as mathematics. It was a research programme with real successes. String lengths in ratio $2:1$ give an octave, $3:2$ a fifth, $4:3$ a fourth. It predicted structure, and the structure was there.

Then it died, killed by its own most famous theorem.

Take a square with side $1$. By Pythagoras famous theorem $a^2 +b^2 = c^2$ its diagonal is $\sqrt2$. The Pythagorean research programme says that number must be a ratio of whole numbers, but it can't be. Therefore not all numbers can be represented as a ratio of whole numbers and we come across the first wrinkle of how we'd like mathematics to behave versus what it actually tells us. The schools most famous theorem produced the very number its philosophy and number system could not explain

The proof you have probably seen

Suppose $\sqrt2 = m/n$ with $m$ and $n$ whole numbers sharing no common factor. Then $m^2 = 2n^2$, so $m^2$ is even, so $m$ is even (an odd number squared is odd). Write $m = 2k$. Then $4k^2 = 2n^2$, so $n^2 = 2k^2$, so $n$ is even too. But $m$ and $n$ were supposed to share no common factor, and they are both now proved to be even, thus having $2$ as a common factor. This is proof by contradiction that $\sqrt2 \ne m/n$.

Short, and slightly unsatisfying. It turns on an arithmetic accident about even numbers and gives no feeling for why the diagonal is out of reach.

The proof that explains it

The move: instead of arguing about factors, show that any whole-number solution manufactures a strictly smaller one. Positive whole numbers cannot shrink forever, so there were none to begin with.

Take a square with side $n$ and diagonal $m$, both assumed to be whole multiples of some common unit, the way $6$cm and $8$cm are both exact multiples of $2$cm. Two lengths related that way are called commensurable. Fold the side onto the diagonal: mark off length $n$ along the diagonal from one corner. The leftover piece has length $m - n$. Build the square on that leftover piece: a short similar-triangles argument shows its diagonal is what remains of the original side once you subtract the leftover a second time, $n - (m-n) = 2n - m$. The important observation is not the geometry itself. It is that the new square has exactly the same side-to-diagonal relationship as the old one, only smaller: side $m-n$, diagonal $2n-m$, same ratio as the original $n$ to $m$.

Fold the side, length $n$, onto the diagonal, length $m$. The leftover piece, $m-n$, becomes the side of a new, smaller square, whose own diagonal is $2n-m$, and the same fold happens again on that square, shown in the middle panel. Both new lengths are whole numbers, both are strictly smaller than $n$ and $m$, and the construction keeps repeating, shown continuing in the third panel. That process never ends. That is impossible for positive whole numbers, so the assumption that started it, that $n$ and $m$ shared a common unit, was false.

This proof says something the first does not. The process just described, take two lengths, replace the larger with the leftover after subtracting the smaller, and repeat on the new pair, is what we now call the Euclidean algorithm. Run it on two commensurable lengths, say $12$ and $8$, and it terminates: $12-8=4$, then $8-4=4$, then $4-4=0$, and the common unit has been found, here $4$. Run it on the side and diagonal of a square and it never terminates. Each round produces a smaller pair with exactly the same shape as the last, so the algorithm keeps finding a remainder, forever, and never lands on zero.

What was actually lost

The repair took two millennia, so it is worth being precise about what actually broke. The Greeks had a fully worked theory of ratio for whole numbers. Once you admit lengths that are not ratios of whole numbers, every theorem stated in terms of proportion becomes untrustworthy, because the objects in the statement are no longer defined. Geometry had magnitudes it could construct and could not name.

Eudoxus supplied the repair around 370 BCE, recorded as Definition 5 of Book V of Euclid. It is one of the great definitions in mathematics and it is almost never taught:

Magnitudes $a, b$ are in the same ratio as $c, d$ when, for every choice of whole numbers $m, n$: whenever $ma > nb$ we also have $mc > nd$, whenever they are equal both are equal, and whenever $ma < nb$ we have $mc < nd$.

Here is the remarkable part. Take every fraction $m/n$ and sort it into one of two piles, depending on whether the ratio $a:b$ is bigger or smaller than $m/n$. Eudoxus's test says two ratios are equal exactly when they produce the same two piles, every time, for every fraction you try. This definition never mentions irrational numbers directly. The ratio is identified not by a number but by where it falls among the fractions, the set below it and the set above it.

That is Dedekind's construction of the real numbers, published in 1872, and it is worth pausing on why the resemblance is not a coincidence. Dedekind's actual construction, in modern language, defines each real number as a cut: a split of every rational number into two groups, the ones below the cut and the ones at or above it, with the cut point itself being whatever number (rational or not) sits exactly on the boundary. $\sqrt2$, for instance, is entirely determined by the split "every negative rational, and every non-negative rational whose square is less than $2$" on one side, and "every non-negative rational whose square is at least $2$" on the other. There is no gap between the two groups, and no rational number sits exactly on the boundary, which is precisely how Dedekind manufactures a new number, $\sqrt2$, to fill it. Eudoxus's Definition 5 does the same split for ratios of magnitudes, two thousand years earlier, without the word "number" ever needing to mean something new. Dedekind said the resemblance was not accidental, that he had, without realising it, rediscovered Eudoxus's idea in a more general form. The idea sat unused for over two thousand years because nobody needed it until analysis started producing limits that had to converge to something.

The property everything rests on

Here is what the rationals lack. Take a set of numbers, $S$, where every member $x \in S$ is a rational number: the rationals themselves do not always contain the object this section is about to build.

Before the formal statement, an easy example, small enough to see all at once. Take every number strictly between $0$ and $2$: the open interval $(0, 2)$, open meaning the two endpoints, $0$ and $2$, are excluded, so the set contains $1.9999$ and $1.99999$ but never quite reaches $2$ itself. (A closed interval, written $[0,2]$, would include both endpoints. The distinction matters here precisely because it decides whether the ceiling you are about to find lives inside the set or just outside it.) What is an upper bound of this set? Any number $u$ with $x \leq u$ for every $x$ in the set, that is, a number sitting at or above every member of it. $2$ works. So does $3$, and so does $100$: none of the numbers in $(0,2)$ ever reach them. A set can have infinitely many upper bounds, and most of them are wasteful, $100$ is a true statement about the set but not a useful one.

Somewhere among all those upper bounds there is a best one, the smallest number that is still an upper bound. For $(0, 2)$, that number is $2$ itself. Every number smaller than $2$ fails to be an upper bound, because you can always find a member of $(0,2)$ above it (if someone proposes $1.999$, the set contains $1.9999$, which is bigger). And $2$ succeeds, because nothing in the set ever reaches it. That best, smallest upper bound is called the least upper bound, or supremum, of the set.

Notice something a little strange: $2$ is the least upper bound of $(0,2)$, even though $2$ is not itself in the set, which is exactly what "open" bought us above. The least upper bound (supremum) does not have to belong to the set it bounds. It only has to be the tightest ceiling available. That is the entire content of completeness: whenever a set of real numbers has an upper bound, there is a unique tightest one, and it is guaranteed to exist as a real number.

The adjective real is not decorative. It is the reason the statement is true. Replace the reals by the rationals and the guarantee disappears. Some sets of rational numbers climb steadily towards a perfect ceiling that is not itself rational. The set has upper bounds, but no least rational one. There is simply a hole where the answer ought to be.

One more example, because the pattern is worth seeing twice before it becomes a formal statement. Take the set $\{1, 1.4, 1.41, 1.414, 1.4142, \ldots\}$, the successive decimal truncations of $\sqrt2$, each one a genuine rational number. Every number in this set is less than $1.5$, so $1.5$ is an upper bound, and so is $2$, and so is $10$. What is the least upper bound (supremum)? It has to be at least as large as every truncation, and no truncation ever exceeds $\sqrt2$ itself, so $\sqrt2$ is a natural candidate, and in fact it is the least upper bound: anything smaller than $\sqrt2$ is eventually beaten by some truncation in the list. But $\sqrt2$ is not a rational number, and every member of this particular set is. If you tried to run this example using only the rationals as your universe, refusing to admit $\sqrt2$ as a legitimate answer, the set would have plenty of upper bounds and no least one, exactly the failure the next paragraph makes precise.

Completeness. Every non-empty set of real numbers that has an upper bound has a least upper bound, and that least upper bound is itself a real number.

Read that as: take any collection of numbers that does not run off to infinity. There is a ceiling above all of them. Completeness says there is a lowest possible ceiling, and that it is a number you are allowed to use.

This is an axiom: a rule accepted as a starting point rather than proved from something more basic. Arithmetic tells us how numbers behave. Completeness tells us which numbers are allowed to exist. It is the defining property of the real numbers, and every construction of them, every recipe for building the reals from scratch, is a way of manufacturing an object that has it. Completeness is what stands between a sequence and a hole: without it, a sequence can spend forever approaching a value that the number system simply does not contain.

And it genuinely fails for fractions. Take all the positive fractions whose square is less than $2$. This set has ceilings: $2$ is one. But it has no lowest fraction ceiling. Whatever fraction you propose, either its square is below $2$, in which case you can nudge it up slightly and still be below, so it was not a ceiling; or its square is above $2$, in which case you can nudge it down and still be above, so it was not the lowest. The rationals have a hole exactly where $\sqrt2$ should be. This is the same set, in different clothes, as the decimal-truncation example above: a collection of rationals with a perfectly good upper bound, and no least one, because the number that should be sitting at the boundary is $\sqrt2$, and the rationals will not admit it.

Every algorithm we'll come across that converges, converges because of completeness. It is assumed every time anyone writes a limit.

Computing a number that does not exist yet

The Babylonians had a method a thousand years before the Pythagoreans had the problem. Guess x. If x is larger than $\sqrt2$, then $2/x$ is smaller than $\sqrt2$​, and vice versa. The two estimates bracket the true value, so averaging them produces a better estimate. This gives the iteration

$$x_{n+1} = \frac{1}{2}\left(x_n + \frac{2}{x_n}\right)$$

Starting from $x_1 = 1$, and keeping every value as an exact fraction to calculate $\sqrt2$:

$n$	$x_n$ as a fraction	as a decimal	error
1	$1$	1.000000000	$-4.1 \times 10^{-1}$
2	$3/2$	1.500000000	$8.6 \times 10^{-2}$
3	$17/12$	1.416666667	$2.5 \times 10^{-3}$
4	$577/408$	1.414215686	$2.1 \times 10^{-6}$
5	$665857/470832$	1.414213562	$1.6 \times 10^{-12}$

The error squares at each step. Three correct digits become six, then twelve.

Quadratic convergence against linear, on the same problem. The averaging step doubles its correct digits each time and hits the limit of double precision by the sixth iterate. Halving the interval gains a fixed amount per step and is still nowhere near.

Why does it converge? Not because the table looks convincing. A table, however long, only ever shows finitely many terms, and no finite amount of evidence can prove what a sequence does forever. The argument instead leans on completeness, and it does so in two separate steps, each one a small, checkable fact about the sequence, that combine into a guarantee.

Step one: the sequence never drops below $\sqrt2$. Take any positive number $x$ and look at the next term the recipe produces, the average of $x$ and $2/x$. Compare it to $\sqrt2$ by computing the difference of its square from $2$:

$$x_{n+1}^2 - 2 \;=\; \left(\frac{x_n^2 - 2}{2x_n}\right)^{2}$$

Read that as: whatever the right-hand side looks like, it is something squared. A squared real number can never be negative, no matter what sits inside the parentheses. So the left-hand side, $x_{n+1}^2 - 2$, can never be negative either, which means $x_{n+1}^2 \geq 2$, which means $x_{n+1} \geq \sqrt2$ (since everything here is a positive number, and among positive numbers, bigger square means bigger number). Every term from the second onward, no matter how the sequence started, sits at or above $\sqrt2$. The sequence has a floor.

Step two: the sequence never goes back up. Compare consecutive terms directly, for any $n \geq 2$:

$$x_n - x_{n+1} \;=\; \frac{x_n^2 - 2}{2x_n}$$

Read that as: the right-hand side is a fraction whose bottom, $2x_n$, is positive (every term in this sequence is positive, by construction), and whose top, $x_n^2 - 2$, is exactly the quantity step one just proved is never negative, for every term from the second onward. A non-negative number divided by a positive one is non-negative. So $x_n - x_{n+1} \geq 0$, meaning $x_n \geq x_{n+1}$: each term is at least as large as the one that comes right after it. The sequence never increases.

Putting the two steps together. The sequence sits above a floor, $\sqrt2$, and it only ever goes down or stays level, never up. A decreasing sequence bounded below cannot continue stepping downward forever without approaching its greatest lower bound. Picture it on a number line: a value that keeps stepping downward but is physically blocked from crossing below $\sqrt2$. Consider the set of all the sequence's values, $\{x_1, x_2, x_3, \ldots\}$. It is bounded below, and by the same completeness axiom applied to the set of values above which everything sits (a short rearrangement turns a floor into a ceiling by flipping every term's sign), this set has a well-defined greatest lower bound, the largest number that is still a floor for the set.

Call that number $L$ and combined with monotonicity (moves only in one direction), this gives us exactly the notion of limit from the start of this piece, made concrete: $L$ is the single value the sequence's terms settle closer and closer to, precisely because they are forced downward, blocked from below, and cannot overshoot back past where they have already been.

Once $L$ exists, its value is easy to pin down. Since $x_{n+1}$ is built from $x_n$ by the same rule at every step, and both $x_n$ and $x_{n+1}$ approach the same limit $L$ as $n$ grows, that rule must hold for $L$ itself:

$$L = \frac{1}{2}\left(L + \frac{2}{L}\right)$$

Read that as: whatever number the sequence is converging to, feeding it into the averaging rule must give the same number back, otherwise the sequence would still be moving. Multiply both sides by $2L$ and simplify, and this single equation reduces to $L^2 = 2$. The limit the sequence converges to is, by definition, a square root of $2$, and since every term in the sequence is positive, the limit is the positive one: $L = \sqrt2$.

Every $x_n$ in that table is a fraction. The limit is not. A sequence living entirely inside the rationals, converging to something outside them, is the whole problem in one line. It is exactly why the rationals are unusable as a setting for analysis.

That template, show the iterates stay somewhere, show something moves monotonically, apply completeness, is the shape of every convergence proof we'll come across. When a solver fails to converge in practice, one of those two conditions has broken, and knowing which one tells you whether to change the starting point or change the method.

Where this breaks in practice

Everything above is about the real numbers. Your machine does not have them.

Almost every modern CPU stores numbers using IEEE 754, and almost all of this series's numbers will be double precision: the standard 64-bit format, giving roughly 15 to 17 correct decimal digits per number. It is a finite set of about $1.8 \times 10^{19}$ values, unevenly spaced: dense near zero, sparse far from it.

Precision is relative, not absolute. Around 1, consecutive doubles differ by about $2\times10^{-16}$. Around 10,000, they differ by about $2\times10^{-12}$. Around $10^{15}$, by $0.125$.

Three consequences that bite.

Addition is not associative. Take $10^{100}$, $-10^{100}$ and $1$. Group the huge pair first and they cancel exactly, leaving $1$. Group the other way and adding $1$ to $-10^{100}$ changes nothing at all, because $1$ is far below the gap between representable numbers out there, so it is discarded before the cancellation happens. Answer: $0$.

The same failure happens with numbers anyone would type: $(0.1 + 0.2) + 0.3$ gives $0.6000000000000001$ while $0.1 + (0.2 + 0.3)$ gives $0.6$. Any algorithm whose correctness argument assumes you can regroup a sum is making an unchecked claim, and that includes anything summing in parallel, where the grouping is chosen by the scheduler.

Cancellation. When two nearly equal numbers are subtracted, their leading digits agree and cancel, so whatever noise was sitting in the trailing digits gets promoted to the front, dominating the result. Numerical analysts call this catastrophic cancellation. The textbook variance formula does exactly this:

$$s^2 = \frac{1}{n-1}\left(\sum x_i^2 - n\bar{x}^2\right)$$

Algebraically correct. Numerically dangerous when the mean is large relative to the spread.

Same data, shifted. With prices around 10,000 and a standard deviation of 0.01, the two terms in the textbook formula agree to roughly twelve significant figures, leaving about four. Welford's algorithm updates the mean and the squared deviations together, so the dangerous subtraction never happens. It is what every production variance estimator actually uses.

The algorithm is named for B. P. Welford¹.

Equality is not a test. x == y for computed floats is a question about bit patterns, not magnitudes. Tolerances are not sloppiness, they are the correct semantics.

Same three numbers as the associativity example above, a different failure this time. Why is $0.1 + 0.2 \neq 0.3$? Not because arithmetic is imprecise. Every one of those roundings is exact and deterministic. It is that the input literals are not the numbers you wrote. Neither $0.1$ nor $0.2$ has a finite binary expansion, both instead repeat forever in base 2, exactly as $1/3$ repeats forever in base 10:

$$0.1_{10} = 0.0001100110011001100\ldots_2$$

Each is rounded to the nearest representable number, meaning a value the machine can actually store, on the way in. This is a statement about which fractions terminate in base 2, not about your hardware.

The connecting idea: completeness guarantees limits exist. Nothing guarantees your machine can represent them, or that the sequence of representable approximations behaves like the sequence in the proof. Every convergence theorem in this series is proved about the map and executed on the terrain. Most of the time the gap does not matter, which is what makes it dangerous. It shows up as a solver that stalls rather than crashes, a variance that comes out negative, an iteration that oscillates forever between two values. None of those are bugs in the mathematics, and none are visible in the proof.

The people

The Pythagoreans were not a faculty. They were a secretive brotherhood with dietary rules, communal property, and a convention of attributing every discovery to Pythagoras himself, which is why we cannot say with confidence who proved anything.

The story everyone tells is that Hippasus of Metapontum revealed incommensurability, the property from earlier of two lengths sharing no common unit, to outsiders, and was drowned at sea for it. Treat it as legend. The earliest surviving version is Iamblichus, writing around 300 CE, some eight centuries after the events, and he is not even consistent with himself: in one telling the drowned man gave away the construction of the dodecahedron, a twelve-faced solid with pentagons for sides and one of the five Platonic solids, and in another the same fate befalls whoever revealed irrationality itself. Ancient writers never actually credit Hippasus with discovering irrational numbers at all; that connection was assembled later, out of two separate half-remembered stories. What the legend does record accurately is that the discovery was experienced as a catastrophe rather than as an interesting result.

Theodorus of Cyrene proved irrationality case by case for $\sqrt3$, $\sqrt5$ and onward, and according to Plato's dialogue Theaetetus stopped at $\sqrt{17}$. Nobody knows why he stopped there. Reconstructing his method from that one clue has occupied historians for a century, on the reasoning that the right method would explain the stopping point.

Eudoxus, who supplied the repair, studied at Plato's Academy while too poor to live in Athens itself. He rented rooms in Piraeus, the port town about seven miles away, and is said to have walked in and back each day rather than miss the lectures.

Dedekind, who finally rebuilt the real numbers in 1872, was unusually candid about where the idea came from: he was teaching calculus, was dissatisfied at having to fall back on geometric intuition for something he could not otherwise justify, and dated the insight to 24 November 1858². He then sat on it for fourteen years before publishing³.

One notable mistake, since this series makes a habit of collecting them. Cauchy's 1821 Cours d'Analyse states as a theorem that a convergent series of continuous functions has a continuous sum⁴. It is false. Abel found a counterexample in 1826, a Fourier series converging to a discontinuous function⁵. The gap is a missing distinction, between converging at each point and converging uniformly across all of them at once, and it was not properly patched until Weierstrass supplied it in the 1870s.

From words to symbols

This whole piece has been deliberately light on notation, because every symbol in it needed its meaning built first, in plain language, before being introduced. Here is the same content again, in the shorthand you will meet in print. None of it is new; it is a translation of what you have already read.

A sequence is written $x_1, x_2, x_3, \ldots$, or $(x_n)$ for short. Saying it converges to a limit $L$ is written

$$x_n \to L \quad\text{as}\quad n \to \infty$$

read as "$x_n$ tends to $L$ as $n$ goes to infinity" i.e. we take later and later terms in the sequence. For example, the Babylonian sequence from earlier, $1, \tfrac32, \tfrac{17}{12}, \tfrac{577}{408}, \ldots$, converges this way, written $x_n \to \sqrt2$.

Irrationality is a membership statement. $\sqrt2$ does not belong to the rationals, $\mathbb{Q}$, but does belong to the reals, $\mathbb{R}$:

$$\sqrt2 \notin \mathbb{Q}, \qquad \sqrt2 \in \mathbb{R}$$

The $\notin$ is what the whole first proof in this piece was for. Everything after it was about the gap that symbol points at.

Completeness itself is usually written with $\sup$, short for supremum, the formal name for least upper bound:

$$\text{if } S \subseteq \mathbb{R} \text{ is non-empty and bounded above, then } \sup S \text{ exists and } \sup S \in \mathbb{R}$$

Read $\sup S$ as "the smallest ceiling over $S$." For the open interval from earlier, $\sup(0,2) = 2$; for the fractions whose square is less than $2$, $\sup\{x \in \mathbb{Q} : x^2 < 2\} = \sqrt2$, exactly the object with no rational name.

$\sup$ is the modern notation for the same thing Definition 5 from 370 BCE was already describing. Eudoxus's test sorted every fraction into two piles, below a ratio and above it, and declared two ratios equal when their piles matched exactly. That is precisely how a supremum is pinned down: a set of rationals below it, a set above it (or equal to it), and nothing rational sitting exactly on the boundary when the object being described is irrational. Two definitions, twenty-two centuries apart, doing the same job: identify a number by where it falls among the rationals, rather than by writing the number down directly.

What to carry forward

1. The rationals are closed under arithmetic but not under limits. That single defect is what makes analysis a subject.
2. Completeness is the property doing the work: every non-empty set of reals with an upper bound has a least upper bound, and that least upper bound is itself a real number. It is not obvious, it is not provable from the other rules of arithmetic, and it is assumed every time you write $\lim$.
3. $\sqrt2$ is irrational because no ratio of whole numbers can equal it, and the rationals are incomplete because a set like "fractions whose square is under $2$" has upper bounds but no least one. Those are the same fact, seen twice: irrationality is what completeness exists to repair.
4. Eudoxus identified a magnitude by how it separates the rationals. Defining an object by its relationship to all the simpler objects around it is a move that keeps coming back.

One sentence to remember: The Greeks discovered a number they could not write down, mathematics answered by enlarging the number system until every sequence that obviously ought to converge actually does and everything in modern analysis begins from there.

---

The companion notebook computes every number in this piece, including the exact binary expansion of 0.1 and the variance comparison, with an associated testsuite.

Footnotes

1. B. P. Welford, "Note on a Method for Calculating Corrected Sums of Squares and Products," Technometrics 4, no. 3 (1962): 419–420. ↩

2. Richard Dedekind, dated the insight to 24 November 1858; see the Dedekind entry collecting his own account. ↩

3. Richard Dedekind, Stetigkeit und irrationale Zahlen (Braunschweig: Friedrich Vieweg und Sohn, 1872). ↩

4. Augustin-Louis Cauchy, Cours d'analyse de l'École royale polytechnique, I.re partie: Analyse algébrique (Paris: Debure frères, 1821); see also MacTutor's summary. ↩

5. Niels Henrik Abel, "Untersuchungen über die Reihe," Journal für die reine und angewandte Mathematik 1 (1826): 311–339. Available via De Gruyter. ↩