Towards a geometric representation of probability distributions.

Over the last few weeks I've been building Arbital, an open-source project exploring a simple question:

Can relationships between variables be represented geometrically rather than reduced to a single coefficient?

Instead of treating measures such as correlation or mutual information as isolated statistics, Arbital builds a polar two-dimensional representation where different types of relationships naturally occupy different regions of the same space.

It has been an interesting project, but it left me with a much deeper question.

If we can describe the relationship between multiple variables geometrically, why can't we do the same for a single probability distribution?

At first glance, this seems like a solved problem.

Statistics gives us an incredibly rich toolbox: moments, quantiles, entropy measures, goodness-of-fit tests, divergences, robust estimators, L-moments, trimmed statistics, rank-based measures, information criteria, kernel estimates and countless other ways of describing data.

But the physicist in me wasn't satisfied.

Those quantities tell us what a distribution looks like, but not necessarily where it belongs.

They feel more like measurements than true coordinates.

I tend to look beyond the measurements and search for the underlying structure. Is there a natural representation, a geometry with symmetries, conserved quantities and meaningful coordinates, that reveal a deeper picture?

That led me to wonder whether probability distributions deserve a second look.

Rather than asking:

"Does this sample follow a Gamma, Lognormal or Student-t distribution?"

could we instead ask:

"Where does this distribution live?"

Imagine every common distribution family occupying its own region within a universal coordinate system.

Instead of comparing distributions pairwise, fitting one model after another, or relying on an ever-growing collection of descriptive statistics, every empirical dataset would simply become a point in the same space.

That immediately raises a more fundamental question.

What should the axes represent? Here's what I've ruled out;

  • Not arbitrary principal components.
  • Not opaque learned embeddings.
  • Not simply another collection of statistical descriptors.

Instead, axes that capture something fundamental about the nature of the distribution itself.

The more I explored the idea, the more I found myself defining the desired properties such a representation should have.

It should be:

Universal

applicable across common distribution families.

Interpretable

every axis should have an intuitive statistical meaning.

Invariant

unaffected by translation, scaling and other transformations that shouldn't alter the underlying character of the distribution.

Continuous

distributions should move smoothly through the space as their parameters change.

Geometric

proximity should reflect genuinely similar statistical behaviour rather than arbitrary numerical similarity.

The more I think about it, the more I realise that statistics has given us an extraordinary collection of ways to describe distributions.

I'm interested in something slightly different.

A way to locate them.

I'm less interested in inventing another statistical measure than in understanding whether there is a deeper geometric representation beneath them all: a universal coordinate system where familiar probability distributions become locations within a common landscape rather than isolated mathematical objects.

Whether such a representation exists is a the question worth exploring.