Elements of Mathematics
From First Principles for Data Scientists, AI Engineers and Quants
What five intimidating concepts actually let you do, before we earn any of them
You are reading a paper, or a textbook, or a colleague's code, and you hit a sentence like this:
we embed the observations in a reproducing kernel Hilbert space and compute the Wasserstein distance between the resulting measures
Every word there is doing real work. None of it is decoration. And if you have not met those words, the sentence is a brick wall.
The usual advice is to go and learn the prerequisites, which means a year of analysis at least before you find out whether any of it was worth it. That is a bad deal, if your ambition is more so application and it is also unnecessary, because the ideas behind those words are not hard. What is hard is the machinery underneath them, and the machinery is not the point.
So this piece gives you the punchlines first. Five scary concepts, given one sentence each, and what each one then lets you do. No proofs. No prerequisites. You will not be able to use any of it afterwards, and that is fine. The goal is that when this series later spends multiple articles building the machinery, you already know what the point is.
There has never been a better time to learn mathematics, and there has never been an easier way to avoid it. Modern software libraries let us train neural networks without understanding optimisation, fit Gaussian processes without learning Hilbert spaces, and compute Wasserstein distances without knowing what a measure is. That is enormously productive, until the day something breaks, a paper introduces a new idea, or you need to build something that nobody has built before.
Every major advance in machine learning and AI eventually leads back to mathematics. Backpropagation became optimisation, attention became linear algebra, diffusion became stochastic calculus, and large language models and embeddings are increasingly understood through information geometry. Yesterday's research paper becomes tomorrow's library, and today's mysterious theorem becomes tomorrow's standard interview question. If you only learn software, you inherit other people's ideas. Learn the mathematics, and you inherit the ability to create your own.
Two promises
Nothing is assumed. Not calculus, not linear algebra, not the Greek alphabet. When a symbol turns up, it gets said at the moment it appears. When a proof makes a leap, the leap gets named rather than performed.
Nothing is dumbed down. This series ends at optimal transport, persistent homology and sheaf Laplacians, which is advanced material by any measure. The accessibility is in the treatment, not the level. Short proofs that teach something are here in full. Long technical ones get the idea, an honest statement that details are being skipped, and a specific book to go to. What you will never get is silence.
Those two promises pull against each other. Most books resolve that by picking a side. This one keeps both, and pays for it in words: plain-English readings of formal statements, pictures, and code you can run to watch a claim be true. At times you will find explanations that seem simple. Bear with them: they were meant for readers newer to this than you.
You might reasonably ask why this chapter begins with Hilbert spaces instead of vectors, or homology before calculus. The answer is that this is not the order in which mathematics was invented, nor even the order in which it is easiest to learn. It is the order in which modern machine learning, quantitative finance and data science quietly rely upon it.
These five ideas appear repeatedly in contemporary research. Kernel methods quietly assume Hilbert spaces. Diffusion models and distribution shift use optimal transport. Bayesian statistics rests on measure theory. Natural gradients live on manifolds. Topological data analysis uses homology to detect structure that ordinary statistics cannot see.
Later chapters will build each idea from the ground up. For now, this chapter simply gives you a map, so that when these words appear again you already know why someone invented them.
Five Concepts
Every mathematical object began life because somebody wanted to solve a problem.
These are five key concepts that people repeatedly encounter when they move from undergraduate mathematics into modern statistics, machine learning and quantitative finance.
Hilbert space
One sentence: a space where you can still measure lengths and angles even though its points are functions rather than arrows, so "find the closest one" still means something.
Imagine every possible curve instead of every possible arrow. You already know the finite version. Fit a line through some points and you are finding the closest line to your data, where closest means the squared errors add up to the smallest total. Square the errors, add them, take the square root: that is a distance, and the fitted line is the nearest point in the space of lines you allowed yourself.
A Hilbert space is that idea when the space of candidates is infinite. Not lines, but functions, or random variables, or infinite sequences. The remarkable part, and it is the spine of this whole series, is that almost nothing changes. The formula is the same. The reason it is optimal is the same. The picture is the same.
What it buys you: conditional expectation, principal components, kernel methods and Fourier analysis stop being four techniques and become one technique applied four times, whether it's a product recommendation, a search engine ranking pages, or an SVM separating two classes. The machinery looks different each time. The underlying mathematics does not.
Manifold
One sentence: a space that looks flat if you stand close enough, like the surface of the Earth, so ordinary calculus works locally even though the whole thing is curved.
Any small patch of the Earth looks like a flat map, and you can navigate a city without ever worrying that the planet is round. But no flat map covers the whole globe without lying somewhere, and that is not a failure of cartography. Gauss proved it is impossible.
This matters because of the phrase "the data lies on a manifold," which gets used loosely and means something precise: your data sits in a thousand-dimensional space but really only varies along a handful of directions, and those directions change depending on where you are. Strong claim. Sometimes true. Often assumed without checking. If your optimisation algorithm assumes the world is flat when it isn't, it wastes steps walking in the wrong directions.
What it buys you: the vocabulary to say why gradient descent gets confused when you rescale your parameters, and what the fix is. Face recognition systems rely on the same fix: photographs with millions of pixels turn out to vary along surprisingly few directions, and natural gradient methods account for exactly the curvature ordinary optimisation ignores.
Measure
One sentence: a rule for assigning sizes to sets, general enough to cover length, area, probability and expectation at once, and robust enough to survive taking limits.
This sounds like the most abstract of the five and yet is the most practical. What is the size of a set? Length answers that for intervals, area for regions, probability for events. One subject masquerading as three, and noticing that is what makes modern probability possible.
It needs a whole theory rather than a definition because the obvious approach breaks. Take well-behaved functions converging to a limit, integrate each one, and ask whether the integrals converge to the integral of the limit. With the integral taught at school: sometimes not, with no warning when it fails.
What it buys you: expectations that behave, and a precise meaning for "information available at time ," the thing every no-arbitrage argument in finance secretly rests on, which is why measure theory underpins stochastic calculus, martingales and almost every mathematical model of derivative pricing.
Homology
One sentence: Homology forgets almost everything about a shape except the features that survive stretching.
Picture a scatter plot forming a ring: a circular band with an empty middle. Every clustering algorithm you run will report some number of clusters, because that is the question clustering asks. None will report the thing that actually characterises the data, which is that it has a hole in it.
Homology counts those holes. Build a skeleton from the points, see which loops can be filled in and which cannot, count the ones that cannot. The answer survives stretching, rotating and rescaling, because it never used distances in the first place.
What it buys you: a description of your data's shape that clustering cannot produce. Two fraud networks with the same number of accounts, transactions and average degree can look identical to ordinary graph statistics; homology asks whether they contain the same holes, loops and persistent structures, sometimes exactly the difference between coordinated behaviour and random activity.
Transport
One sentence: the cheapest way to move one pile of stuff into the shape of another, where you get to say what cheap means.
You have two distributions and want to know how different they are. Comparing means misses most of it. KL divergence reports infinity when they do not overlap. Neither tells you how far apart they are.
Optimal transport asks differently. Treat each distribution as a pile of sand. What is the least total work to move the first into the shape of the second, where moving a grain further costs more? That minimum is a real distance, finite even when the distributions share no common ground, and it degrades gracefully rather than catastrophically.
What it buys you: a distance between distributions that works when the standard ones do not, which is why diffusion models, domain adaptation and distribution alignment all lean on it to measure how one probability distribution should move to become another.
The one idea underneath
If you take one thing from this, take this:
Four of these ideas repeatedly reduce to one recurring mathematical move: find the best approximation inside a constrained space.
Not literally. Structurally. The pattern:
You have something you want. You cannot have it, because it is not in the set of things available to you. So you find the closest available thing, and you can prove it is closest because the error points in a direction you could not have used anyway.
That is projection. It is fitting a line, in ordinary space with arrows. It is the same thing in a space whose points are functions. It is conditional expectation, where the thing you want is a random variable and the things available are the ones you could actually know at the time. It is the separating hyperplane, where the available set is a convex region rather than a flat subspace. It is the representer theorem, inside the space a kernel builds. It is loops in a graph.
Once you recognise projection, you begin seeing it everywhere. Linear regression projects observations onto the space of straight lines. Fourier analysis projects functions onto waves. PCA projects data onto lower-dimensional directions. Conditional expectation projects random variables onto the information currently available. Kernel methods project data into richer spaces where simple geometry becomes useful again.
At first these look like unrelated algorithms. By the end of this series I hope they look like the same argument told in different dialects.
This is not twenty plus topics. It is about the same ideas, reframed and restated until they are unmistakable.
The others worth watching for:
| The move | First seen | Last seen |
|---|---|---|
| Project onto what you can afford | Pythagoras | kernels and graphs |
| Nothing is linear, but everything is linear locally | tangent lines | curvature and natural gradients |
| Throw away what deforms away, count what survives | counting corners | persistent homology |
| Before you can average, decide what size means | ratios | Wasserstein distance |
One worked example, so this is not all talk
Three observations. A predictor taking values ; a response taking values . Fit a line through the origin: find the number making as close to as possible.
Most people are taught to write down the total squared error, differentiate, set to zero, solve. That works, and it needs calculus.
The way this series will teach you: notice that and are two arrows in three-dimensional space, that multiples of the first form a line, and that you want the point on that line closest to the tip of the second. Drop a perpendicular:
Read that fraction out loud, because the notation is doing something simple. The top multiplies the two lists entry by entry and adds the results: , measuring how much the arrows point the same way. The bottom does the same to with itself: , measuring how long is. Dividing answers "how many copies of do I need," and makes the answer independent of the units was measured in.
No calculus. And the geometric route tells you something the calculus route does not: the leftover error is exactly perpendicular to . Which means the residuals of a fitted model are uncorrelated with its predictors as a matter of algebra, not as evidence the model is any good.
If you have ever plotted residuals against a predictor, seen no pattern, and felt reassured, that is worth sitting with. The plot could not have come out any other way.
Why this matters
If you're a data scientist
Most models fail for reasons that have nothing to do with Python and everything to do with mathematics. Why did your regression overfit? Why does PCA work? Why is covariance enough here but not there? Why do kernels behave like infinite-dimensional linear models? Mathematics gives you the language to answer those questions instead of treating libraries as magic.
If you're an AI engineer
Today's AI architectures are built by combining ideas from optimisation, probability, geometry and information theory. The field moves quickly, but the mathematics moves slowly. Learn the underlying ideas once, and new architectures become easier to understand because you recognise the patterns rather than memorising another acronym.
If you're a quant
Modern quantitative finance is almost entirely mathematics expressed as software. Derivative pricing depends on stochastic calculus. Portfolio optimisation depends on geometry. Risk models depend on probability and linear algebra. Market making increasingly depends on optimal transport and information geometry. The notation changes, but the recurring ideas do not.
If you're simply curious
Mathematics is one of humanity's oldest ways of compressing complicated ideas into simple ones. Once you recognise the recurring patterns, papers become shorter, proofs become less intimidating and entirely different subjects begin to look strangely familiar.
What this will not do
It will not make you a mathematician. You will not come out proving new theorems. You will come out able to read a theorem's statement, know what it claims, know roughly why it is true, and know whether it applies to your situation. Different skill, more useful for most people who need this material.
It will not pretend the hard parts are easy. Where something is difficult, it says so.
It will not skip things silently. Every omitted proof is named as omitted, with a reference.
It will not tell you the map is the terrain. Every technique here is a map, and every map distorts. Real numbers are a map of magnitude and your computer cannot properly represent them. A covariance matrix is a map of dependence that throws away any relationship which is not pairwise. Where the distortion is knowable in advance, currently biting, and checkable, this series stops and tells you what to check.
Three questions to hold onto
No exercises: there is nothing yet to practise. But the series will answer these three, and you will know it is working when the answers start to seem obvious.
- Why should fitting a line, computing a conditional expectation, and running a support vector machine turn out to be the same operation?
- Your data has a hole in it. What could you compute that would tell you so, when every clustering algorithm will instead report a number of clusters?
- Two distributions have no overlap at all. In what sense can one such pair be further apart than another?
One sentence to remember: you are not learning twenty-three topics, you are learning about a few key ideas that keep coming back wearing different clothes, and the first is that the best available answer is the closest available answer.
Most mathematics books teach definitions and then examples. This book attempts the reverse. We will begin with the problems that forced people to invent new mathematics, then build the ideas slowly enough that, by the time the formal definitions appear, they feel inevitable rather than mysterious.
Next: the number that broke arithmetic, where a group of Greek mathematicians proved something they did not want to be true, it took 2,200 years to repair, and the same problem is why your computer thinks .