The Hidden Geometry of Generative AI: How Manifold Theory Explains the Mysteries Nobody Explained to You

---

Introduction

Why does a diffusion model have to destroy an image with noise before it can create one?

Why did GANs collapse, oscillate, and make grown engineers cry for five straight years?

Why does king – man + woman = queen actually work inside a language model?

Why can a tiny printed sticker fool a vision system that outperforms trained radiologists?

Why can you slide a dial in “latent space” and watch one face morph smoothly into another?

Why doesn’t deep learning drown in the curse of dimensionality – when every statistics textbook says it should?

Six mysteries.

Every practitioner has bumped into them. Almost nobody can explain them.

I’m writing this article to tell you that there is one mathematical idea that explains all six – and a seventh bonus mystery I’ll throw in for free.

That idea is manifold theory.

Read this article to the end to find out why.

And here’s my promise to you: every single concept in this piece gets explained twice.

Once in plain English, for every curious human being on this planet.

Once in the style of the great popular mathematics writers – the real theorems, the real names, the real ideas, told as stories rather than symbols.

Not one formal equation appears anywhere in this article – the only arithmetic you will meet is done on words.

You choose your depth.

Nobody gets left behind.

Sound fair?

Read on!

---

The One Idea That Explains Everything

Here is the claim, stated as simply as I can state it:

Your data does not fill the space it lives in. It lies on a thin, curved, low-dimensional surface inside that space – a manifold. Generative AI is the art of finding that surface, moving along it, and never falling off.

That’s it.

That is the manifold hypothesis – and once you truly see it, you cannot unsee it.

Every mystery in this article is a direct consequence of that single sentence.

But what is a manifold?

What Is a Manifold? (In Plain English)

Take a sheet of paper.

A sheet of paper is a two-dimensional object.

Every point on it can be described with just two numbers – how far along, how far up.

Length and width. That’s it.

Now crumple that sheet and toss it into the middle of your room.

The paper now lives in three-dimensional space.

It bends, folds, and curves through your room.

But has the paper itself changed?

No – it is still, intrinsically, a two-dimensional thing.

An ant walking on its surface still only needs two numbers to know where it is.

The room is the ambient space.

The crumpled sheet is the manifold.

Now scale that picture up until it hurts!

A single 224 x 224 colour photograph is a list of 150,528 numbers – one number per pixel per colour channel.

So every photo is a single point in a 150,528-dimensional room.

Here is the thought experiment that changed how I see AI forever.

Imagine picking those 150,528 numbers at random.

What do you get?

Static. Television snow. Noise.

Pick again.

Static.

Again.

Static.

You could keep picking random points in that room for the entire age of the universe – trillions of times over – and you would never, not once, produce a photograph of a cat.

Why?

Because real photographs – cats, faces, sunsets, cathedrals – do not fill that 150,528-dimensional room.

They live on a whisper-thin, curved, folded surface inside it.

A crumpled sheet in a cosmic room.

Almost all of the room is static.

The world, as a camera sees it, is the sheet.

And here is the number that should make you fall out of your chair: researchers have actually measured the dimension of that sheet for ImageNet photographs, and the estimates land somewhere between roughly 25 and 45 intrinsic dimensions.

(Pope et al., ICLR 2021, https://arxiv.org/abs/2104.08894)

Not 150,528 dimensions.

About forty.

A forty-dimensional crumpled sheet floating in a 150,528-dimensional room.

Remember this!

That ratio is the secret engine of everything that follows!

What Is a Manifold? (For the Math-Curious)

Now let me tell you how the mathematicians see it – and I promise you will enjoy this, because their formal definition is secretly a story about maps.

Ask a geometer what a manifold is, and the soul of the answer is: any shape that looks flat if you zoom in far enough.

The Earth is a sphere – but stand in a field outside Chennai and the ground around you looks perfectly flat.

Every point of the sphere has a little patch around it that could pass for a piece of ordinary flat paper.

A shape with that property everywhere – locally flat, globally curved however it pleases – is a manifold.

And how do mathematicians work with such a shape?

Exactly the way explorers worked with the Earth: they build an atlas.

Each page of a world atlas is a flat map of one region.

No single flat page can capture the whole curved globe – but a book of overlapping pages can, provided the pages agree with each other wherever they overlap.

That is, almost word for word, the formal definition of a manifold: a collection of local flat maps, called charts, stitched together consistently.

Cartography, promoted into a definition.

I love mathematics for things like this.

Three more ideas, each one a jewel:

1. Every curved world fits inside a flat room.
   • Hassler Whitney proved, back in the 1930s and 40s, that any abstract curved space – no matter how wildly it twists – can always be placed inside ordinary flat space, and you never need more than twice its own number of dimensions to do it.
   • So “a low-dimensional sheet sitting inside a big flat room” is not a convenient picture I chose for you.
   • It is a theorem about every manifold there is.
   • John Nash – yes, the Beautiful Mind himself – later showed you can even do the placing without distorting a single distance.
2. Give the sheet a ruler, and shortest paths appear.
   • Bernhard Riemann’s great idea (1854): let every point of a surface carry its own tiny ruler and protractor.
   • Suddenly you can measure the length of any curved path along the surface – and ask which path between two points is the shortest.
   • Those champions are called geodesics, and you already know one: it is why your Chennai-to-San-Francisco flight arcs up toward the pole instead of following the straight line on a wall map.
   • Remember geodesics. They star in Case Study 3.
3. Every sheet has a safety margin.
   • Herbert Federer asked a wonderfully practical question: how far can you drift away from a surface before “which point of the surface am I closest to?” stops having one clear answer?
   • That distance is the surface’s reach – a single number measuring how tightly the sheet is folded.
   • A gently rolling ribbon has a generous reach; crumpled foil has almost none.
   • Remember reach too. It stars in Case Studies 1 and 5.

The manifold hypothesis then says: natural data – images, audio, sensor streams – concentrates on or near such a sheet, whose own dimension is tiny compared to the room’s.

And this is not a vibe.

Fefferman, Mitter and Narayanan turned it into a hypothesis you can statistically test on real data, with rigorous guarantees (“Testing the Manifold Hypothesis”, Journal of the AMS, 2016, https://arxiv.org/abs/1310.0425).

And the measuring instruments keep returning the same verdict: for image collections whose ambient rooms have hundreds of thousands of dimensions, the sheets measure in the tens (Pope et al., 2021: MNIST digits, about 7 to 13; ImageNet photographs, roughly 25 to 45).

One honest caveat before we proceed, because I refuse to oversell you: real data is better described as lying near a manifold – a sheet wearing a little fuzz – and possibly as a union of several sheets with different dimensions.

I will give this objection a full and fair hearing later in this article. Hold that thought.

Now – the mysteries.

Case Study 1: Why Diffusion Models Destroy Before They Create

Stable Diffusion. Midjourney. DALL-E. FLUX.1.

Every one of them is built on the same bizarre ritual: take perfectly good training images, drown them in Gaussian noise step by step until nothing remains, and then teach a network to run the film backwards.

Destruction first. Creation second.

For years, this struck newcomers as insane.

Why would adding noise be the key to generating photorealistic images?

Manifold theory answers in one breath.

In Plain English

Imagine you want to teach a robot to find its way to a mountain ridge – the ridge is our data manifold, the thin surface where all real images live.

The natural tool is a compass that always points “toward more realistic”.

In diffusion language this compass is called the score – the direction of steepest increase in probability.

Here’s the fatal problem.

If all real images live on a whisper-thin sheet, then at almost every point in the giant room, the probability of real data is exactly zero – and it’s zero in every direction around you too.

Your compass needle just spins.

Zero here, zero there, zero everywhere.

No gradient. No signal. No direction home.

The compass only works on the sheet – and you’re never on the sheet.

So what does noise do?

Noise inflates the sheet.

Blur every training image with a little Gaussian noise, and the razor-thin manifold puffs up into a fuzzy cloud around itself.

Add more noise, the cloud swells further.

Add enough, and the fuzz fills the entire room.

Now the compass has a reading at every single point in space – because every point is at least somewhat close to the inflated data cloud.

And now the generation trick becomes obvious:

Start from pure static – anywhere in the room.

Follow the compass through the thickest fog, where it works everywhere.

As you get closer, dial the fog down, and follow the sharper compass of a thinner cloud.

Fog level by fog level, you are guided all the way down onto the bare manifold itself – onto a real image.

That is a diffusion model.

Noise is not the enemy.

Noise is the scaffolding that makes the compass readable across the whole room.

You destroy the image during training so that, at generation time, you can find your way home from anywhere.

Beautiful, isn’t it?

For the Math-Curious

Here is how the mathematicians say “the compass has no reading” – and it is worth hearing, because the problem is deeper than it sounds.

Probability in a big room is measured by density – how much probability lives per unit of room.

And a whisper-thin sheet, having zero thickness in almost every direction, holds its probability the way a sheet of paper holds air: everything squeezed into zero volume.

Ask “what is the density of real images at this point of the room?” and the honest answer is zero everywhere except exactly on the sheet – where it is, in effect, infinite.

The score – the compass that points toward higher density – works by comparing densities at neighbouring points.

Zero next to zero next to zero.

Nothing to compare.

So off the sheet, the compass is not merely unreliable.

It is undefined.

Song and Ermon put their finger on precisely this manifold obstruction, and then built the modern diffusion model around the cure (“Generative Modeling by Estimating Gradients of the Data Distribution”, 2019, https://arxiv.org/abs/1907.05600).

The cure, in the mathematicians’ language, is blurring.

Mix every data point with a puff of Gaussian noise, and the razor-thin distribution smears into a smooth fog with honest, positive density at every point of the room.

Now the compass reads everywhere.

And here comes my favourite twist – the reason training these models is even practical.

Pascal Vincent noticed in 2011 that you never need to measure the compass directly.

You only need to teach a network to denoise – to guess the clean image hiding underneath the noise.

Why is that enough?

Because of a gorgeous old piece of statistics called Tweedie’s formula, which says: compare your best guess of the clean signal with the noisy point you are standing on, and the arrow between them points exactly along the direction of increasing probability.

The denoiser IS the compass.

Every “guess the noise” exercise is secretly a compass-calibration exercise.

A mid-century statistical trick, waiting seventy years for its moment.

Stack a whole ladder of fog levels – thick, thinner, thinnest – and you get the “annealed” sampling of Song and Ermon.

Let the ladder become a continuum and you get the grand modern picture (Song et al., 2021, https://arxiv.org/abs/2011.13456): the forward destruction of an image is a movie governed by one differential equation – and Brian Anderson proved, back in 1982, that every such movie has a mathematically exact rewind button, and that the rewind needs only one input to work: the compass reading at every fog level.

DDPM, the model that started the craze (Ho et al., 2020, https://arxiv.org/abs/2006.11239), is the frame-by-frame version of the same film.

And the geometric punchline that I find genuinely stunning.

At low fog, the learned compass needles all point perpendicular to the sheet – straight back home, the shortest way in.

So stand near the sheet and count how many independent directions the needles actually use: you are counting the directions that lead off the sheet, and the directions the needles ignore are the ones that run along it.

Stanczuk and colleagues turned this into a genuine measuring procedure (“Your diffusion model secretly knows the dimension of the data manifold”, 2022): interrogate a trained diffusion model’s compass field, count carefully, and out comes the dimension of your dataset’s sheet.

Your image generator is, quite literally, a manifold-measuring instrument.

Nobody planned that. The geometry was there all along.

Case Study 2: Why GANs Made Grown Engineers Cry

Between 2014 and 2019, Generative Adversarial Networks were the kings of image generation – and the most miserable models on Earth to train.

Vanishing gradients. Wild oscillation.

And the infamous mode collapse: you train on ten thousand kinds of cats, and the generator proudly produces the same three cats forever.

The community threw a decade of tricks at the problem.

But the diagnosis – the actual mathematical reason GANs suffered – came from manifold theory.

And it is gorgeous.

In Plain English

A GAN is a forger and a detective.

The forger (generator) paints fake images; the detective (discriminator) tells fake from real.

They train against each other.

Now bring back our picture: real images live on one whisper-thin sheet in the giant room.

The forger’s fake images live on another whisper-thin sheet – because the forger is itself a smooth machine painting a low-dimensional surface.

Question: what are the odds that two whisper-thin sheets, floating in a room of a hundred and fifty thousand dimensions, happen to lie exactly on top of each other?

Zero. Effectively, absolutely zero.

Two hairs floating in an ocean.

And if the two sheets don’t overlap, the detective’s job is trivial.

It can slide a wall between them and be 100% correct, everywhere, forever.

Fake on this side.

Real on that side.

Done.

Here’s the tragedy: a perfect detective is a useless teacher.

All it ever says is “wrong!” – with total confidence and zero direction.

It never says “warmer” or “colder”, because there is no “warmer” when the verdict is already absolute.

The forger receives no usable feedback.

Its learning signal flatlines.

That is the vanishing gradient problem – not a bug in the code, but a geometric inevitability of two thin manifolds that don’t intersect.

And mode collapse?

Also geometry – this time, topology.

The forger paints its sheet as one continuous, connected surface – one unbroken piece of fabric, because smooth machines cannot tear.

But real data is often an archipelago: the island of tabby cats, the island of Siamese cats, the island of Persians, with empty ocean – impossible images – between them.

You cannot cover an archipelago with one unbroken sheet without draping fabric across the ocean – without generating garbage images between the islands.

The forger’s least-bad option?

Camp on one island.

Paint tabby cats. Only tabby cats.

Forever.

That is mode collapse. Not a mystery.

A theorem wearing a trench coat.

For the Math-Curious

Arjovsky and Bottou wrote the diagnosis down with full rigour (“Towards Principled Methods for Training Generative Adversarial Networks”, 2017, https://arxiv.org/abs/1701.04862), and here is what their theorems say in honest English.

Mathematicians keep several classical gauges for the question “how different are these two probability distributions?” – the Kullback-Leibler divergence, the Jensen-Shannon divergence.

Beautiful instruments.

And every one of them shares the same blind spot: they only compare the two distributions where they overlap.

Hand them two whisper-thin sheets that never touch – which, for two low-dimensional sheets floating in a vast room, is essentially guaranteed – and the gauges peg at their maximum reading and freeze there.

Kullback-Leibler reads infinity.

Jensen-Shannon reads its absolute ceiling.

And here is the killer: the reading is identical whether the fake sheet is a millimetre from the real one or a mile away.

The original GAN, the theorems show, was training against exactly this Jensen-Shannon gauge.

A needle pegged at maximum carries no information about which way to move – and “no information about which way to move” is, precisely, a vanished gradient.

Five years of GAN misery, diagnosed in one line: the loss function was a smoke alarm that screams at the same volume whether the fire is in your kitchen or on another continent.

The rescue was to swap the gauge for one that feels distance.

The Wasserstein distance – the “earth mover’s distance” – asks a different question entirely: what is the cheapest total effort required to shovel one pile of probability into the exact shape of the other?

Move the fake sheet a millimetre closer and the shovelling bill drops by a millimetre’s worth.

The needle moves smoothly the whole way in.

Gradients everywhere.

That is the Wasserstein GAN (Arjovsky, Chintala and Bottou, 2017, https://arxiv.org/abs/1701.07875).

And the trick for actually computing it is a jewel from Leonid Kantorovich – the mathematician whose theory of optimal allocation later won a Nobel in economics.

Kantorovich’s duality says: instead of planning all the shovelling yourself, hire a critic.

Let the critic assign a score to every point of the room, trying to score real points high and fake points low – under one strict rule: the critic’s scores are never allowed to change too fast from place to place.

A speed limit on opinions.

The theorem guarantees that the best score-gap any rule-abiding critic can achieve equals the shovelling cost exactly.

So the discriminator of a Wasserstein GAN is reborn as a speed-limited critic – the speed limit enforced crudely at first by clipping the critic’s weights, then properly with a gradient penalty (Gulrajani et al., 2017, https://arxiv.org/abs/1704.00028).

Mode collapse, meanwhile, has a one-line topological core.

A continuous machine cannot tear fabric – and unbroken fabric, projected from one connected blob of latent space, arrives as one connected piece.

So if the real data is an archipelago – several disconnected islands – the generator faces a forced choice: drape garbage across the open sea, or camp on a subset of the islands.

A theorem wearing a trench coat, exactly as promised.

Khayatkhoei and colleagues (NeurIPS 2018) took the hint literally and built GANs with several generators – one sheet per island.

Notice the pattern forming.

The pathology was geometric.

The cure was geometric – retire the gauges that are blind to the distance between sheets, and hire the one gauge that measures it.

Optimal transport will return in Case Study 7 to collect its crown.

Case Study 3: The Soap Bubble in Latent Space

You’ve seen the demos.

A slider moves, and a young face ages smoothly into an old one.

A sedan morphs into an SUV.

Winter becomes summer over the same lake.

This is latent space interpolation – and it hides two mysteries, not one.

Mystery A: why does sliding through latent space produce meaningful changes at all, instead of scrambled garbage?

Mystery B: why do straight-line morphs sometimes pass through blurry, mushy, low-quality frames in the middle?

Both answers are geometric.

The second one involves a soap bubble that genuinely shocked me the first time I understood it.

In Plain English

A generator is a map-projector.

The latent space is the map – a tidy, compact chart, say 512 dials.

The image manifold is the territory – the crumpled sheet in the cosmic room.

The generator’s entire job is to project every point of the map onto the territory.

That answers Mystery A immediately.

When you drag a point across the map, its projection glides along the crumpled sheet – along the surface of real images.

You never leave the world of plausible pictures, because the projector, by construction, only ever prints points on the sheet.

Meaningful interpolation isn’t a miracle.

It is what “moving along a manifold” looks like.

Mystery B is subtler, and the explanation is one of my favourite facts in all of high-dimensional geometry.

Ask yourself: where do random points in a 512-dimensional Gaussian cloud actually live?

Intuition says: bunched around the centre, like a fuzzy ball.

Intuition is dead wrong.

In 512 dimensions, essentially all of the probability mass concentrates in a thin shell at distance of about the square root of 512 – roughly 22.6 units – from the centre.

Almost nothing lives near the middle.

The “cloud” is a soap bubble: a hollow sphere with a thin skin and a vast, empty interior.

Your model was trained on latent points drawn from that bubble – which means it only ever learned to project the skin.

Now interpolate between two points in a straight line.

Two points on opposite sides of a bubble… and your straight chord dives through the hollow interior – through a region the projector has never seen in its life.

Result: the mushy, low-quality middle frames.

The fix, once you see the bubble, is obvious: don’t tunnel through the bubble – walk along its surface.

That is slerp, spherical linear interpolation (popularised for generative models by Tom White, “Sampling Generative Networks”, 2016, https://arxiv.org/abs/1609.04468).

Curved path on the map, clean images all the way.

Straight lines are a flatlander’s habit.

High-dimensional space punishes flatlanders.

For the Math-Curious

Mystery A, in the mathematicians’ telling, is almost an anticlimax – which is exactly the point.

The generator is a smooth machine from a small flat space into the giant room, and the output of such a machine is a low-dimensional surface sitting inside the room.

Drag a point across the map and you drag its projection along that surface.

Interpolation is manifold traversal not by luck but by construction.

Case closed.

Mystery B is where high-dimensional probability gets gloriously weird.

Take a bell-curve cloud in 512 dimensions and ask: how far from the centre do random points actually land?

Each of the 512 coordinates contributes its own small random amount, and 512 small random contributions average out – the same law of large numbers that makes a casino’s monthly income boringly predictable even though every single bet is pure chance.

The result: the total distance from the centre comes out almost exactly the same for every point you draw – about 22.6 units, the square root of 512 – with a wobble of less than one unit either way.

All of the probability, concentrated in one thin shell.

The soap bubble is not a metaphor.

It is the law of large numbers wearing geometry’s clothes.

Now run the straight chord between two points on that bubble.

At its midpoint, the distance to the centre has dropped to about 16 – roughly nine wobble-widths below the shell.

How rare is it for an honest random draw to land that deep inside?

So rare that you could keep drawing from now until the stars burn out and never see it happen once.

The model has never been asked about that region – and your straight-line morph is asking it to paint there.

Hence the mush.

Slerp repairs it by travelling along the shell at constant distance from the centre, like a ship following the curve of the Earth instead of tunnelling through the mantle.

But there is a deeper and more beautiful statement waiting here, and it is the one I want you to keep.

The latent map’s own ruler is a lie.

The honest distance between two nearby points on the map is not measured on the map at all – it is measured by how far apart their projected images land on the sheet.

In effect, the generator hands the map a new, warped ruler: distances stretch wherever the projector stretches, and compress wherever it compresses.

And the moment a space carries a position-dependent ruler, it has – by Riemann’s very definition – a geometry of its own, complete with its own shortest paths.

Arvanitidis, Hansen and Hauberg computed those honest shortest paths inside real deep generative models and showed two things: the latent spaces are genuinely curved, and interpolating along the honest geodesics gives visibly better morphs and statistics than straight coordinate lines (“Latent Space Oddity: on the Curvature of Deep Generative Models”, ICLR 2018, https://arxiv.org/abs/1710.11379).

Tom White’s slerp, from 2016, was the practical first taste of the same medicine (https://arxiv.org/abs/1609.04468).

Read that again: the right way to move through a deep generative model is differential geometry, verbatim.

Rulers, curvature, geodesics.

Riemann died in 1866.

His mathematics is rendering faces in 2026.

Case Study 4: King − Man + Woman = Queen. But WHY?

In 2013, Mikolov and colleagues at Google dropped a result that still feels like a magic trick (https://arxiv.org/abs/1301.3781).

Train a simple model to embed words as vectors.

Take the vector for king.

Subtract man.

Add woman.

Search for the nearest word vector.

You get queen.

No grammar rules.

No dictionary of analogies.

Nobody programmed this.

Arithmetic on learned coordinates somehow does semantics.

Thirteen years later, this same phenomenon – scaled up a million-fold – is quietly running inside every large language model you talk to.

And manifold geometry explains it.

In Plain English

Here is the picture to hold in your mind: on the manifold of meanings, concepts are directions.

Somewhere in the embedding space there is a direction that means “male → female”.

Start at king, walk along that direction, and you arrive at queen.

Start at actor, walk the same direction, arrive at actress.

Start at Paris, walk the “country-of” direction, arrive at France.

King − man + woman = queen is just: “find the male→female arrow using one pair of words, then apply that same arrow to a different starting point.”

Vector arithmetic works because meaning, locally, is organised along consistent directions on the representation manifold – like north and east on a map.

Now the objection you should be raising: a language model knows millions of concepts.

Its embedding space has only a few thousand dimensions.

You cannot fit millions of perpendicular arrows into a few thousand dimensions!

Correct – you cannot fit millions of perfectly perpendicular arrows.

But here is the high-dimensional miracle: you can fit an astronomically large number of almost perpendicular arrows.

In high dimensions, “nearly perpendicular” directions are absurdly abundant – the number of directions you can pack while keeping them all almost-orthogonal grows exponentially with the dimension.

A few thousand dimensions comfortably hosts millions of concept-arrows with only whisper-level interference between them.

This packing trick even has a name in the interpretability world: superposition.

Anthropic’s researchers demonstrated it in toy models (“Toy Models of Superposition”, 2022, https://transformer-circuits.pub/2022/toy_model/index.html), then went hunting inside a production model – and in 2024 they extracted millions of interpretable feature directions from Claude 3 Sonnet, including one now-legendary direction that, when amplified, made the model obsess over the Golden Gate Bridge in every answer it gave.

One direction.

One concept.

Turn the dial, change the mind.

Meaning is geometry.

I told you the sheet was everywhere.

For the Math-Curious

The modern name for the phenomenon is the linear representation hypothesis: inside a model’s representation space, high-level concepts behave – locally, approximately – like directions, so that combining concepts is adding arrows, and intensifying a concept is lengthening its arrow (for the careful modern treatment, including the surprisingly subtle question of what “perpendicular” should even mean in these spaces, see Park, Choe and Veitch, 2023, https://arxiv.org/abs/2311.03658).

The capacity puzzle – millions of concepts, only thousands of dimensions – is resolved by one of the most counterintuitive facts in all of geometry, a cousin of the celebrated Johnson-Lindenstrauss lemma.

In our familiar three dimensions, you can fit exactly three mutually perpendicular arrows, and that is that.

But relax “perpendicular” to “almost perpendicular” – allow each pair of arrows a whisker of overlap – and high dimensions explode with room: the number of nearly-perpendicular arrows you can pack grows exponentially with the dimension.

A few thousand dimensions can host not thousands, not millions, but astronomically many almost-independent directions, each pair interfering only at whisper level.

That packing trick is superposition, and Anthropic’s interpretability team caught it in the act.

First in miniature, in transparent toy models where you can watch a network deliberately store more features than it has dimensions by tilting them into nearly-perpendicular arrangements (“Toy Models of Superposition”, 2022, https://transformer-circuits.pub/2022/toy_model/index.html).

Then in the wild: using sparse autoencoders – think of them as prisms that split the model’s mixed internal light back into its pure component colours – they pulled millions of interpretable feature-directions out of a production model, Claude 3 Sonnet, in 2024 (“Scaling Monosemanticity”).

Including one now-legendary direction which, when artificially amplified, made the model steer every conversation – every single one – toward the Golden Gate Bridge.

One direction. One concept. Turn the dial, change the mind.

Two honest footnotes, because precision matters more than romance:

1. Word-analogy arithmetic is approximate and famously curated; it works well for some relations and poorly for others.
2. The geometry is local and statistical, not a law of physics.
3. Raw text is a string of discrete symbols – there is no smooth surface of sentences, no such thing as “slightly more than a word”.
4. The manifold structure lives in the embedding and activation spaces the network builds for itself, not in the raw tokens.
5. That distinction matters, and I will give it its full due in the objections section, where it belongs.

But the headline stands, and it is a genuine scientific discovery of our era: the internal world-model of a large language model is, to a striking first approximation, a geometric object – concepts as directions, composition as addition, and the whole library folded into fewer dimensions than it has ideas by the exponential generosity of high-dimensional space.

And that, to me, is the first time I really understood how vector embeddings work!

And, as an extension, how Generative AI works.

Manifolds are everywhere.

Case Study 5: The Sticker That Breaks a Superintelligence

Here is the humiliation that launched a thousand research careers.

Take a state-of-the-art image classifier – one that beats humans on benchmark after benchmark.

Show it a panda. It says “panda”, 99% confident.

Now add a perturbation so tiny that you cannot see any difference at all.

The model says “gibbon”.

With even higher confidence!

Szegedy et al. discovered this in 2013 (“Intriguing Properties of Neural Networks”, https://arxiv.org/abs/1312.6199), and the field has never fully recovered.

Printed stickers fool object detectors.

Specially patterned glasses defeat face recognition.

How can systems this capable be this fragile?

Manifold theory offers the single most clarifying lens I know – with one honesty clause I’ll state up front: unlike the previous case studies, this one is a powerful partial explanation, not the whole verdict.

I’ll show you the counter-evidence too.

You deserve both.

In Plain English

Go back to our numbers.

Natural images: a roughly 40-dimensional sheet.

Ambient pixel room: 150,528 dimensions.

Now stand on the sheet and count your escape routes.

At every single point, there are about 40 directions that move you along the sheet – toward other real images – and roughly 150,488 directions that step off it into the void of impossible images.

The off-ramps outnumber the roads by nearly four thousand to one.

Your classifier made you a promise: “I behave sensibly on the sheet” – because the sheet is the only place it ever saw training data.

Off the sheet?

No promises. No training.

Terra incognita.

Its decision boundaries slice through that void in wild, arbitrary, untested ways.

An adversarial attack is simply this: take a tiny step along many of those 150,488 off-sheet directions at once.

Each component is microscopic – invisible to your eye, which only perceives sheet-level structure.

But their combined effect teleports the point off the world the model knows, into void-regions where its boundaries say whatever they were never trained not to say.

The panda didn’t become a gibbon.

The panda left the universe.

And a tiny step was enough – because when the exits outnumber the roads four thousand to one, the void is never more than a whisker away.

For the Math-Curious

The organising idea is codimension – the count of missing dimensions.

A 40-dimensional sheet in a 150,528-dimensional room is missing 150,488 of them, and every missing dimension is a perpendicular escape direction available at every single point.

The classifier’s training data never visited those directions, so nothing ever disciplined its behaviour out there; its decision boundaries carve up the void arbitrarily, unsupervised and untested.

An attacker with a tiny total budget can spend a microscopic amount along thousands of escape directions at once – each contribution invisible on its own, the sum a teleport.

And the tighter the sheet is folded – the smaller its reach, Federer’s safety margin from earlier – the shorter the step needed to leave the world entirely.

Supporting evidence that this picture carves reality at a joint: Stutz, Hein and Schiele (“Disentangling Adversarial Robustness and Generalization”, CVPR 2019) explicitly sorted adversarial examples into on-sheet and off-sheet varieties – and found that many standard attacks are off-sheet, that on-sheet adversarial examples behave like ordinary generalisation errors, and that defending against the two requires different medicine.

Two diseases, one symptom.

Now the honesty clause, delivered as promised:

1. Goodfellow, Shlens and Szegedy (2015, https://arxiv.org/abs/1412.6572) blamed the fragility primarily on models being too linear in high dimensions – thousands of microscopic nudges, each multiplied by a weight and then summed, add up to one decisive shove.
2. An explanation about the function, not the sheet.
3. Gilmer and colleagues built the cleanest possible test world – data living on two perfect, fully known spheres – and found that even superbly trained models with essentially no test error still had adversarial examples lying ON the data surface (“Adversarial Spheres”, 2018, https://arxiv.org/abs/1801.02774).
4. The culprit there is concentration of measure again – the soap-bubble law from Case Study 3: in high dimensions, almost every point of a sphere sits one tiny step away from some mistake, the way almost every point on a vast frozen lake is inches from a hairline crack.
5. Geometry gives the explanation; geometry also complicated it.
6. Ilyas and colleagues, “Adversarial Examples Are Not Bugs, They Are Features” (2019, https://arxiv.org/abs/1905.02175), showed that the perturbations often exploit patterns which are genuinely predictive in the data – real signal, just signal no human eye can perceive.
7. Part of the phenomenon lives on-distribution, not off it.

My assessment, stated plainly: the off-sheet, codimension picture explains a large share of the phenomenon – above all, why invisible perturbations can work at all – but it is one pillar of a multi-cause story, alongside model linearity and imperceptible-but-real features.

High confidence in the geometry as a mechanism; moderate confidence in it as the dominant mechanism. That is the honest state of the science in 2026.

Case Study 6: The Curse That Became a Blessing

Open any classical statistics textbook and you will find a horror story called the curse of dimensionality: the number of samples required to learn a function grows exponentially with the dimension of its input.

Take that seriously for images.

To learn a function of 150,528 input variables by classical nonparametric means, you would need more training samples than there are atoms in the observable universe.

Not “a lot”.

Physically, cosmically impossible.

And yet ImageNet has about 1.3 million labelled images – and deep networks learned it.

Modern models train on billions of images and master the visual world.

According to the textbook, deep learning should not exist.

So why does it?

In Plain English

Because the curse charges you for the size of the room – but you only ever needed to learn the sheet.

Learning a function means covering its input territory with enough examples that no query lands too far from something you’ve seen.

Covering a 150,528-dimensional room is cosmically impossible, yes.

But nobody asked you to.

Every image that will ever matter – every photo any camera will ever take – lives on the roughly 40-dimensional sheet.

Cover the sheet and you have covered the world.

And covering a 40-dimensional surface is merely hard, not impossible.

Millions of samples hard.

Billions for the fine details.

Exactly the scale of the datasets we actually have.

The curse of dimensionality was real.

The manifold hypothesis is the loophole.

Deep learning drove through it.

That single loophole is why the entire AI revolution is physically possible at all.

No low-dimensional manifold – no ChatGPT, no Midjourney, no Claude.

The universe’s data had to be compressible for intelligence to be learnable.

Let that sink in.

For the Math-Curious

Statisticians can do something wonderful here: they can compute the exchange rate between dimensions and data.

The classical results of nonparametric statistics say, in effect: to learn a reasonably smooth function to a given accuracy, the number of samples you need grows explosively – and the growth charges you per dimension of the territory you must cover.

That per-dimension surcharge, written out as a price list, IS the curse of dimensionality.

The escape is a family of theorems with one shared punchline: the bill depends on the sheet, not the room.

When data lives on a low-dimensional surface inside a big space, the achievable learning rates are governed by the surface’s own dimension – the ambient dimension drops out of the bill entirely.

This was established first for classical local methods – nearest-neighbour and kernel estimators that quietly adapt to hidden manifold structure (Bickel and Li, 2007; Kpotufe’s line of work) – and then, crucially for our story, for deep networks themselves: deep ReLU networks provably achieve learning rates that depend only on the intrinsic dimension, adapting to the unknown sheet without ever being told it exists (Nakada and Imaizumi, JMLR 2020; Chen and colleagues, 2019, in the same spirit as Schmidt-Hieber’s results on hidden compositional structure).

Now run the numbers and feel the force of it.

Price the job for a learner facing the full 150,528-dimensional room, at a modest fifty-percent accuracy target, and the bill that comes back has tens of thousands of digits – not a big number, a number with no physical meaning in this universe.

Price the same job for the 40-dimensional sheet, and the bill comes out to roughly two million samples.

Two million.

ImageNet has 1.3 million.

The theory’s estimate and the dataset that actually cracked computer vision land within a factor of two of each other.

Yes, this is a back-of-envelope calculation that ignores constants and picks a friendly accuracy target – I won’t pretend otherwise.

But when a statistical scaling law and a historical engineering milestone shake hands even at the order-of-magnitude level, pay attention.

I don’t believe that’s a coincidence.

I believe that’s the geometry drawing the boundary of the possible – and our era happening to be the one that crossed it.

Case Study 7: Flow Matching – Geometry Strikes Again

One more mystery – the bonus I promised – and this one is about the models you are using right now in 2026.

Stable Diffusion 3. FLUX.1.

The current generation of image models quietly abandoned classic diffusion for something called flow matching and rectified flows.

Generation that once took a thousand small steps now takes a handful.

What changed?

Once again: somebody stopped fighting the geometry and started using it.

In Plain English

Remember how a diffusion model gets home: it follows the compass through fog, step by tiny step, along a wiggly, drunken, random-walk path from static to image.

It works – but the road is absurdly long, because the road was inherited from the noise process, and noise does not walk in straight lines.

Flow matching asks the question a geometer would ask on day one:

Why follow the drunkard’s path when you can pave a straight highway from the noise cloud to the data manifold?

So that’s what it does.

Pick a noise point. Pick a data point.

Draw the straight line between them.

Train the network to output, at every point in space and time, the velocity of the traffic flowing along those lines.

Generation becomes: start at noise, drive the highway.

Straight roads need far fewer steps than drunken wiggles.

That is the whole trick – and it’s why a FLUX-class model can produce an image in a handful of steps instead of a thousand.

Same destination manifold. 10X better road.

(And as someone who runs local AI on a modest Dell with 2GB of VRAM, I can tell you: fewer steps is not an abstraction. Fewer steps is the difference between “possible on my machine” and “not”.)

The wonderful irony: optimal transport – the mathematics of moving one pile of earth into the shape of another pile as cheaply as possible – already rescued GANs in Case Study 2.

Now it returned, ten years later, to redesign diffusion itself.

In this field, the transport geometers always collect twice.

For the Math-Curious

Flow matching rests on one law and one absurdly simple trick.

The law is the continuity equation – the traffic law of probability.

It says that probability behaves like a conserved fluid: it can flow from place to place, but it can never leak away and never appear from nowhere.

Prescribe how the fluid should flow – a wind field filling the whole room, telling every droplet of probability which way to drift – and the fog’s shape at every future moment is completely determined.

So the entire design problem collapses into one sentence: choose a wind that carries the featureless noise-fog into the exact shape of the data-fog.

The trick is the training target (Lipman et al., 2023, https://arxiv.org/abs/2210.02747).

Pair up one noise point with one data point, and declare their path to be the straight line between them.

What is the correct wind along that path?

Destination minus starting point.

That’s it. That single arrow – “from here, toward there” – is the entire lesson, repeated across millions of pairs.

No compasses, no fog ladders, no rewind buttons: teach a network the arrows, then ride the wind home.

Rectified flow (Liu, Gong and Liu, 2022, https://arxiv.org/abs/2209.03003) adds one more move, called reflow: retrain on your own generated pairs, and the slightly bent traffic straightens further with every pass, toward genuinely straight lines – which is exactly what makes a handful of steps, even one big step, enough.

Stable Diffusion 3 validated the recipe in production-grade models (“Scaling Rectified Flow Transformers for High-Resolution Image Synthesis”, Esser et al., 2024, https://arxiv.org/abs/2403.03206), and Black Forest Labs’ FLUX.1 family runs on the same flow-matching foundation.

The link back to Case Study 2 is optimal transport in its moving-picture form.

Benamou and Brenier proved that the earth-mover’s cost between two fogs can be recast as a question about winds: among all winds that carry fog A into fog B while obeying the traffic law, the transport cost equals the least total kinetic energy any of them needs.

The cheapest morph is the laziest wind – and the laziest wind moves every droplet in a straight line.

Straightness is not an engineering hack.

It is what optimality looks like in the geometry of probability.

And one jewel to close the case study, because it is too beautiful to omit.

Jordan, Kinderlehrer and Otto proved in 1998 that the equation governing diffusion itself – the spreading of heat, the drifting of ink, the dissolving of an image into noise – is secretly a descent: it is exactly what you get when disorder rolls downhill as fast as it possibly can, with “downhill” measured not by an ordinary ruler but by the earth-mover’s geometry.

Diffusion models were doing optimal-transport geometry all along, in disguise.

Flow matching just took off the mask.

Is the Manifold Hypothesis Even True?

Hold on, Thomas, you might say.

You’ve spent a few thousand words explaining everything with one hypothesis.

That’s suspicious. One idea that explains everything usually explains nothing.

Is the manifold hypothesis actually true?

Fair challenge.

So here is the case against – given honestly, at full strength:

1. Real data is a union of manifolds, not one manifold.

• Brown, Caterini, Ross, Cresswell and Loaiza-Ganem tested this directly (“Verifying the Union of Manifolds Hypothesis for Image Data”, ICLR 2023) and found that image datasets are better modelled as unions of manifolds with different intrinsic dimensions – the “cats” component and the “cathedrals” component do not even have the same dimensionality.
• The single-sheet picture is a simplification. (Notice, though: this is a refinement of the geometric view, not a refutation – and it independently confirms the disconnectedness that Case Study 2 blamed for mode collapse.)

2. Data lies near manifolds, not on them.

• Sensor noise, compression artifacts, film grain – real datasets are manifold-plus-fuzz.
• Where curvature is high and reach is small, “the manifold” becomes a genuinely blurry notion, and estimated dimensions vary across the dataset and across estimators.
• The 25-to-45 ImageNet figures I quoted are estimates with real spread, not constants of nature.

3. Text is discrete.

• Token sequences form a finite combinatorial set – there is no smooth surface of sentences, no tangent space at the word “cat”.
• For language, the manifold structure genuinely lives in the learned embedding and activation spaces – a manifold the model builds, rather than one the raw data possesses.
• That is a philosophically important distinction, and sweeping it under the rug would be dishonest.
• Geometry-of-representations and geometry-of-data are cousins, not twins.

4. Some structure may be fractal or dimension-varying.

• These are objects for which classical smooth-manifold language is simply the wrong vocabulary.
• So where does that leave us?
• Exactly where George Box left every scientist:”

The manifold hypothesis is a model.

It is wrong in the ways just listed – and it is the single most useful wrong idea in modern machine learning.

It correctly predicted why GANs fail, why noise saves diffusion, why straight-line interpolation stumbles, why tiny perturbations fool giants, and why learning is possible at all.

A hypothesis that keeps paying out like that has earned its seat at the head of the table – refinements, unions, fuzz and all.

That is how real science holds ideas.

Firmly, and with open hands.

---

The Geometry Was Always There

Step back with me now, because there is something bigger here than machine learning.

Bernhard Riemann built the geometry of curved manifolds in the 1850s – abstract, unwanted, applauded by almost nobody.

Sixty years later it turned out to be the exact language of Einstein’s spacetime.

A century after that, it turned out to be the exact language of artificial imagination.

Nobody designed neural networks to do differential geometry.

We stumbled into these architectures chasing benchmarks – and found charts, geodesics, transport maps and curvature waiting for us inside them, uninvited.

The mathematics was not applied to the territory – it was the territory.

Four hundred years ago, one man saw this coming:

Triangles, circles, and other geometric figures.

Galileo was writing about the heavens.

But I am a man of faith as well as a man of technology, and I will tell you plainly what I see: the same Author who wrote geometry into the orbits of the planets wrote a quiet, merciful compressibility into the fabric of our world.

Reality did not have to be learnable.

The sheet did not have to be thin.

And yet it is – thin enough that creatures like us, and now machines built by creatures like us, can read the book.

That is not a proof of anything.

It is an observation offered from the bottom of my heart: I find it very hard to look at a forty-dimensional sheet of meaning folded into a hundred-and-fifty-thousand-dimensional void and see an accident.

You may see otherwise.

The geometry, either way, remains – and it is glorious.

Look at what was hiding under the hood of the machines all along –

Curvature.

Charts.

Geodesics.

Transport.

Flow.

Shape.

Meaning.

The universe wrote its book in geometry – and for the first time in history, our machines are learning to read it out loud.

What a time to be alive!

All the very best – and watch this space!

Cheers!

---

References

Every paper, book and classic mentioned in this article, in order of appearance – because you deserve to check my work.

Where an entry carries no link, the title itself is the search key; the canonical version is one query away.

Papers

1. Pope et al. (2021). The Intrinsic Dimension of Images and Its Impact on Learning. ICLR 2021. https://arxiv.org/abs/2104.08894
2. Fefferman, Mitter and Narayanan (2016). Testing the Manifold Hypothesis. Journal of the American Mathematical Society. https://arxiv.org/abs/1310.0425
3. Song and Ermon (2019). Generative Modeling by Estimating Gradients of the Data Distribution. NeurIPS 2019. https://arxiv.org/abs/1907.05600
4. Vincent (2011). A Connection Between Score Matching and Denoising Autoencoders. Neural Computation.
5. Song et al. (2021). Score-Based Generative Modeling through Stochastic Differential Equations. ICLR 2021. https://arxiv.org/abs/2011.13456
6. Anderson (1982). Reverse-Time Diffusion Equation Models. Stochastic Processes and their Applications.
7. Ho, Jain and Abbeel (2020). Denoising Diffusion Probabilistic Models. NeurIPS 2020. https://arxiv.org/abs/2006.11239
8. Stanczuk et al. (2022). Your Diffusion Model Secretly Knows the Dimension of the Data Manifold.
9. Arjovsky and Bottou (2017). Towards Principled Methods for Training Generative Adversarial Networks. ICLR 2017. https://arxiv.org/abs/1701.04862
10. Arjovsky, Chintala and Bottou (2017). Wasserstein GAN. ICML 2017. https://arxiv.org/abs/1701.07875
11. Gulrajani et al. (2017). Improved Training of Wasserstein GANs. NeurIPS 2017. https://arxiv.org/abs/1704.00028
12. Khayatkhoei et al. (2018). Disconnected Manifold Learning for Generative Adversarial Networks. NeurIPS 2018.
13. White (2016). Sampling Generative Networks. https://arxiv.org/abs/1609.04468
14. Arvanitidis, Hansen and Hauberg (2018). Latent Space Oddity: on the Curvature of Deep Generative Models. ICLR 2018. https://arxiv.org/abs/1710.11379
15. Mikolov et al. (2013). Efficient Estimation of Word Representations in Vector Space. https://arxiv.org/abs/1301.3781
16. Park, Choe and Veitch (2023). The Linear Representation Hypothesis and the Geometry of Large Language Models. https://arxiv.org/abs/2311.03658
17. Elhage et al. (2022). Toy Models of Superposition. Anthropic. https://transformer-circuits.pub/2022/toy_model/index.html
18. Templeton et al. (2024). Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet. Anthropic, published on transformer-circuits.pub.
19. Szegedy et al. (2013). Intriguing Properties of Neural Networks. https://arxiv.org/abs/1312.6199
20. Goodfellow, Shlens and Szegedy (2015). Explaining and Harnessing Adversarial Examples. ICLR 2015. https://arxiv.org/abs/1412.6572
21. Stutz, Hein and Schiele (2019). Disentangling Adversarial Robustness and Generalization. CVPR 2019.
22. Gilmer et al. (2018). Adversarial Spheres. https://arxiv.org/abs/1801.02774
23. Ilyas et al. (2019). Adversarial Examples Are Not Bugs, They Are Features. NeurIPS 2019. https://arxiv.org/abs/1905.02175
24. Bickel and Li (2007). Local Polynomial Regression on Unknown Manifolds.
25. Kpotufe (2011). k-NN Regression Adapts to Local Intrinsic Dimension. NeurIPS 2011.
26. Nakada and Imaizumi (2020). Adaptive Approximation and Generalization of Deep Neural Network with Intrinsic Dimensionality. JMLR.
27. Chen et al. (2019). Nonparametric Regression on Low-Dimensional Manifolds Using Deep ReLU Networks.
28. Schmidt-Hieber (2020). Nonparametric Regression Using Deep Neural Networks with ReLU Activation Function. Annals of Statistics.
29. Naitzat, Zhitnikov and Lim (2020). Topology of Deep Neural Networks. JMLR.
30. DiCarlo and Cox (2007). Untangling Invariant Object Recognition. Trends in Cognitive Sciences.
31. Lipman et al. (2023). Flow Matching for Generative Modeling. ICLR 2023. https://arxiv.org/abs/2210.02747
32. Liu, Gong and Liu (2022). Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow. https://arxiv.org/abs/2209.03003
33. Esser et al. (2024). Scaling Rectified Flow Transformers for High-Resolution Image Synthesis. https://arxiv.org/abs/2403.03206
34. Jordan, Kinderlehrer and Otto (1998). The Variational Formulation of the Fokker-Planck Equation. SIAM Journal on Mathematical Analysis.
35. Brown et al. (2023). Verifying the Union of Manifolds Hypothesis for Image Data. ICLR 2023.

Books and classics

36. do Carmo. Differential Geometry of Curves and Surfaces.
37. Lee. Introduction to Smooth Manifolds.
38. Villani. Topics in Optimal Transportation.
39. Riemann (1854). On the Hypotheses Which Lie at the Foundations of Geometry. The habilitation lecture that started it all.
40. Galileo Galilei (1623). The Assayer.

All images AI-generated by NightCafe Studio.