Nobody Knows How LLMs Work – Unless You Look at Them as Non-Linear Dynamical Systems

The nonlinear science behind large language models, explained in plain English, with the numbers that matter – and where it leads next

---

---

Nobody Knows How These Things Work

Ask an honest AI researcher how large language models work, and you will get an uncomfortable answer.

“We know how to build them. We do not fully know how they work.”

That is not false modesty.

It is the actual state of the field.

We can write down the architecture.

We can write down the training loop.

We can tell you every matrix multiplication in exhaustive detail.

And yet nobody can tell you, from first principles, why a model at 15 billion parameters can do arithmetic when a model at 10 billion cannot.

Why does “think step by step” work?

Why do capabilities appear in sudden jumps rather than smooth climbs?

Why does a model that has clearly memorized its training data suddenly, thousands of steps later, start actually understanding?

I’m writing this article to tell you that these questions are not unanswerable.

They only look unanswerable because most people are looking at LLMs through the wrong lens.

Stop looking at a language model as a very complicated program.

Start looking at it as a nonlinear dynamical system.

Do that, and the fog lifts.

The sudden jumps become phase transitions – the same mathematics that governs water turning to ice.

The mysterious plateaus become competitions between internal circuits.

The sensitivity of long reasoning chains becomes the butterfly effect.

The architecture tricks that “just work” turn out to be criticality-engineering devices borrowed, unknowingly, from the physics of sandpiles and earthquakes.

This is published, measurable, verifiable  science – and it comes with real numbers.

Read on!

---

What Is a Nonlinear Dynamical System, Anyway?

Let’s define the term, because everything in this article depends on it.

A dynamical system is anything that changes over time according to rules.

The weather. A pendulum. A stock market. A heart. A population of rabbits.

Linear means effects are proportional to causes.

Push twice as hard, go twice as far.

Double the input, double the output.

Linear systems are predictable, well-behaved, and – frankly – rare in nature.

Nonlinear means the system feeds back on itself.

Output becomes input.

Small causes can produce enormous effects, and large causes can produce nothing at all.

The response is not proportional to the push.

That single property – feedback – is what makes reality interesting.

And nonlinear dynamics splits into two great research traditions, which are the two halves of this article:

Chaos theory studies how simple deterministic rules produce unpredictable behaviour. There is no randomness in a chaotic system. It is fully determined. But it is so exquisitely sensitive to its starting conditions that prediction becomes practically impossible. That is why we can know the exact physics of the atmosphere and still not forecast rain three weeks out.

Complexity theory studies the opposite miracle – how vast numbers of simple parts, interacting locally, spontaneously produce sophisticated global order that nobody designed. Ant colonies. Immune systems. Cities. Brains.

Chaos is order producing apparent randomness.

Complexity is randomness producing apparent order.

An LLM does both. At once. At enormous scale.

Now here’s your first pair of terms.

ChatGPT user:

• An attractor is the pattern a system settles into.
• A pendulum’s attractor is hanging straight down.
• Your model has attractors too – which is exactly why it sometimes gets stuck repeating the same phrase forever.
• That loop is not a bug in the code.
• It is the system falling into a low-energy pattern and being unable to climb out.

AI engineer:

• A strange attractor is the signature of chaos: a bounded region the system never leaves, but within which it never exactly repeats.
• Long autoregressive generation lives on one.
• This is why your outputs are neither random nor reproducible across small perturbations – and why fixing a seed feels like it should give determinism but often does not once you change anything else in the stack.

Hold onto that.

We will need it.

---

The Boring Part That Makes the Exciting Part Possible

The foundational fact of LLM science is the scaling law.

In 2020, OpenAI published Scaling Laws for Neural Language Models, showing that model quality improves in a smooth, predictable curve as you add parameters, data, and computing power.

Parameters are the adjustable dials inside the model.

GPT-3 had 175 billion. Each one gets tuned during training.

More dials means more capacity to store patterns.

Loss is a score for how badly the model predicts the next word.

Lower is better.

It is the only thing training actually optimizes – everything else is a side effect.

Tokens are chunks of text, roughly three-quarters of a word each.

A 300-page book is about 100,000 tokens.

The equation, refined by DeepMind’s Chinchilla paper, is:

Loss = E + A/N^α + B/D^β

In English:

1. Your error is an unavoidable floor (E – language is genuinely unpredictable sometimes),
2. Plus a penalty for being too small (N = parameters),
3. Plus a penalty for reading too little (D = data).

That is the whole industry in one line.

And notice its shape.

It is a power law – and power laws are the fingerprint of nonlinear systems everywhere in nature.

ChatGPT user:

• A power law means a system looks the same at every scale.
• Earthquakes follow one – a few huge ones, many medium, countless tiny, in fixed proportion.
• So do city sizes, word frequencies, and wealth.
• When you see one, you are almost always looking at a system with feedback rather than a system of independent parts.
• AI improvement follows the same statistical signature as earthquakes.
• That is not a coincidence.

AI engineer:

• Power laws are why you can forecast a training run’s final loss from its first few percent – and why gains feel relentlessly expensive.
• To halve your loss gap you multiply compute by a large constant factor.
• The straight line on the log-log plot is your budget planner and your reality check, both at once.
• Deviations from that line are your most valuable diagnostic signal.

The 20-to-1 rule

DeepMind trained over 400 models to find the right balance.

The answer: roughly 20 tokens of training text per parameter.

A 70-billion-parameter model wants around 1.4 trillion tokens.

Why does this matter?

Because before Chinchilla, everyone got it wrong.

GPT-3 used only about 1.7 tokens per parameter – it was starved of data.

Chinchilla-70B beat the 280-billion-parameter Gopher while using four times less computing power, purely by rebalancing.

ChatGPT user:

• This is why newer small models feel as smart as older giants.
• An 8B model in 2026 outperforms a 175B model from 2020 – not because it has more dials, but because it read far more before the exam.

AI engineer:

• This is your cost lever.
• When choosing between a big model and a small well-trained one, check the training tokens, not just the parameter count.
• A well-fed 8B running on your own hardware often beats an underfed 70B on someone else’s API – at a fraction of the inference bill.

Smooth curves. Predictable costs. Everybody happy.

And then the smooth curves broke!

---

Emergence: When the Curve Shatters

In 2022, Google researchers published Emergent Abilities of Large Language Models and found something the scaling laws did not predict.

The loss improves smoothly.

But actual skills do not.

Skills stay at zero.

Then they appear, almost fully formed.

Ability	Appears around
Basic zero-shot task following	~1.5 billion parameters
Two and three-digit arithmetic	~13 billion parameters
In-context learning (learning from prompt examples)	~2.5-5 billion training tokens
Chain-of-thought reasoning becoming useful	~60-100 billion parameters
Complex multi-step logical reasoning	~540 billion parameters (PaLM era)

This is the language of physics, and physics has precise words for it.

ChatGPT user:

• A phase transition is a sudden total change in behaviour from a tiny change in conditions.
• Water does not gradually become ice – it stays liquid, stays liquid, and then at exactly 0°C, snap.
• Your AI models do the same thing.
• Nothing, nothing, nothing – then arithmetic.
• This is why AI progress feels like it comes in shocking leaps rather than steady improvement.
• It genuinely does.

AI engineer:

• The vocabulary matters operationally.
• A control parameter is the knob you turn (scale, data, loss).
• An order parameter is what you measure to detect the transition (accuracy above chance, circuit formation).
• A critical point is where the order parameter goes non-analytic.
• Choosing a bad order parameter is how you miss a transition entirely – which is precisely the technical heart of the “emergence is a mirage” debate below.

Look closely at row three of that table.

That threshold is not measured in model size at all – it is measured in how much text the model has read so far.

We will come back to it.

It is the most beautiful result in the field.

ChatGPT user:

• This explains why “think step by step” works on modern models and did nothing on older or smaller ones.
• Below the threshold, asking for reasoning actively hurts – the model produces plausible-sounding nonsense steps.
• Above it, the same prompt unlocks genuine multi-step problem solving.
• The prompt did not change.
• The model crossed a line.

AI engineer:

• Never assume a technique validated on a frontier model degrades gracefully on a smaller one.
• It may not degrade – it may vanish.
• Test prompting strategies separately on every tier you deploy.
• A chain-of-thought pipeline that works beautifully on a large model can silently collapse when you swap in a cheaper one to cut costs.

“But Thomas – isn’t this all a measurement illusion?”

Fair challenge, and a serious one.

In 2023, Stanford researchers published Are Emergent Abilities of Large Language Models a Mirage?.

Their argument: if you grade with a harsh all-or-nothing metric, you manufacture fake cliffs.

A model getting steadily better at arithmetic looks like zero right up until it is perfect.

They were partly right.

Some emergent abilities do melt away under gentler grading.

But not all of them.

Sharp jumps survive even under smooth continuous measurements – and the biggest transition of all is visible directly in the loss curve, which is about as continuous as measurement gets.

The critique did not kill the field. It made it rigorous.

That is exactly how science is supposed to work!

---

The Real Control Knob Is Not Size

Here is the insight I find most elegant in all of this.

The thresholds above are written in parameters.

But parameters are not what causes the jump.

Pretraining loss is.

Think of it this way.

Loss is the temperature.

Parameters and data are just the stove settings you use to get there.

Water freezes at 0°C whether you cooled it in a freezer, a cold room, or a mountain cave.

The route does not matter.

The temperature does.

So an ability does not “appear at 13 billion parameters.”

It appears when prediction error drops below a critical value – and 13 billion parameters happened to be how you got there in 2022.

ChatGPT user:

• A universality class is one of physics’ most astonishing ideas: totally different systems – a magnet, a fluid, an alloy – can share identical mathematics at their critical points, because only the deep structure matters, not the ingredients.
• This is why the same capability thresholds keep showing up across models built by rival labs with different data and different architectures.
• They are the same kind of system.

AI engineer:

• Stop using parameter count as your capability proxy in procurement decisions.
• It was never the right variable.
• Benchmark against your actual task, because two 7B models can sit on opposite sides of a capability threshold depending entirely on how they were trained.
• Pretraining loss, where disclosed, is a far better predictor than the number on the model card.

And this explains something that puzzles everyone.

ChatGPT user:

• This is why AI keeps getting cheaper without getting dumber.
• Better data and better methods push models past the same capability thresholds at smaller sizes.
• The capability line is fixed.
• We keep finding shorter routes to it.

---

Grokking: Watching a Model Understand, in Slow Motion

Now for the strangest experiment in machine learning.

In 2022, researchers trained tiny models on modular arithmetic – clock math, where 9 + 5 = 2 on a 12-hour clock.

They published Grokking: Generalization Beyond Overfitting.

The model memorized the training examples fast.

Perfect scores on questions it had seen.

Zero ability on new ones. Classic cramming.

Then the researchers kept training. And training.

For thousands more rounds while absolutely nothing changed.

Then – suddenly – the model generalized perfectly.

It had stopped memorizing and started actually understanding.

Memorize.

Plateau.

Snap.

Understand.

Why does this happen?

Because two strategies compete inside the network.

Researchers call it circuit competition.

Strategy A – the lookup table.

Memorize every answer. Fast to learn, but bulky. It eats capacity.

Strategy B – the algorithm.

Actually work out the rule. Slow to discover, but tiny and elegant – in this case, the model literally invents a Fourier-transform-based method.

Training applies gentle pressure toward simplicity.

Early on, memorization wins because it is quicker.

But as training grinds on, the elegant algorithm – far cheaper to maintain – eventually outcompetes and replaces it.

The plateau is not stagnation.

It is a war being fought inside the weights.

ChatGPT user:

• A bifurcation is the moment a system’s behaviour splits into a qualitatively new regime – one stable state becoming two, or a steady state becoming an oscillation.
• Turn up the heat on a pot of water and convection cells appear all at once at a specific temperature.
• Grokking is a bifurcation.
• That is why understanding arrives as an event rather than a gradual dawning.

AI engineer:

• Two practical consequences.
• First, in fine-tuning, a validation plateau does not always mean “stop” – sometimes the model is mid-reorganization.
• Second, and bigger: your dataset size determines which strategy wins.
• Below a critical data size, fine-tuning teaches memorization and you ship brittle models that fail off-distribution.
• Above it, you get generalization.
• If your fine-tune is fragile, more diverse data usually beats more epochs.
• That critical data size grows with model size.

And the theory made falsifiable predictions that came true – including ungrokking, where a model that already generalized reverts to memorizing if you shrink its training set.

When a theory predicts a bizarre new phenomenon and experiment confirms it, that theory is doing real work.

A 2024 unification paper then showed this single mechanism explains three separate mysteries at once: grokking, the double-descent curve, and emergent abilities at scale.

Three puzzles.

One explanation.

ChatGPT user:

• This is the difference between a model that memorized an answer and one that understands a method.
• Understanding is literally simpler inside the network than memorization.
• That is why models handle problems they have never seen – they found the rule, not the answer key.

Generalization is compression.

Occam’s Razor, enforced by gradient descent.

---

The Edge of Chaos: Why Deep Networks Must Live Dangerously

Time for chaos theory proper.

Imagine whispering into one end of a very long chain of people.

Two possible failures:

Too much damping – each person mumbles slightly quieter, and by person fifty the message is silence. Information dies. The ordered phase.

Too much amplification – each person exaggerates slightly, and by person fifty you have gibberish. Small differences explode. The chaotic phase.

Neither chain can carry a message.

But there is a razor-thin setting in between – the edge of chaos – where signals travel any distance without dying or exploding.

Deep neural networks are exactly this chain, with layers instead of people.

And a landmark NeurIPS 2004 paper established that only networks near this critical boundary can perform genuinely complex computation.

Later work made it mathematical.

There is a precise critical line in the settings used to initialize a network’s weights, and information survives arbitrarily deep only exactly on that line.

ChatGPT user:

• Sensitive dependence on initial conditions is the butterfly effect – the idea that a butterfly flapping its wings in Brazil could set off a tornado in Texas.
• The technical measure is the Lyapunov exponent: positive means nearby starting points fly apart exponentially, negative means they converge. Zero is the edge of chaos.
• When one early word flips your model’s whole answer, you are watching a positive Lyapunov exponent in action.
• This is classic chaos theory.

AI engineer:

• This reframes hyperparameter tuning as finding the critical corridor rather than hunting a magic number.
• Long reasoning chains are chaotic systems: a single early token flips the trajectory.
• That is the butterfly effect in your production logs, and the mitigation is architectural – self-consistency sampling, intermediate checkpointing, verifier steps – not prompt-tweaking.

Here is the part that should make you sit up.

Every architectural trick in the modern transformer – LayerNorm, residual connections, careful initialization – exists to hold the network at this edge.

Residual connections give signals a clean path through.

Normalization acts like cruise control, constantly nudging the network back to critical.

Without them, deep transformers suffer rank collapse, where every token’s representation converges to the same thing – the ordered phase’s information death, in production.

We did not design these components as chaos-theory devices.

But that is exactly what they are.

ChatGPT user:

• Self-organized criticality is nature’s habit of tuning itself to the edge.
• Drop sand grain by grain onto a pile: it builds up until it reaches exactly the critical slope, then maintains that slope through avalanches of every size.
• Nobody sets the angle.
• The system finds it.
• Forest fires, earthquakes and neural avalanches in real brains all do this.

AI engineer:

• Normalization layers are an engineered version of exactly that feedback mechanism.
• This is why removing LayerNorm from a deep stack is catastrophic rather than merely suboptimal – you are not losing a small regularization benefit, you are losing the controller that keeps the network on the critical surface where information can propagate at all.

Every trick in the modern transformer playbook is, secretly, Per Bak’s sandpile wearing a lab coat.

How cool is that?

And it explains the temperature slider you use every day.

ChatGPT user:

• This is the science behind temperature.
• Near zero puts generation in the ordered phase – repetition loops, the model saying the same phrase forever.
• Too high puts it in the chaotic phase – fluent, confident nonsense.
• Interesting coherent output lives in the narrow band between.
• Creativity is a critical phenomenon.

---

The Most Beautiful Number in the Field

I saved this one for last.

Anthropic researchers studied 34 transformer models during training and published In-Context Learning and Induction Heads.

In-context learning is the model’s ability to learn from examples inside your prompt.

Show it three examples of a format, and it follows that format on the fourth.

No retraining.

This is the single ability that makes prompt engineering possible at all.

They found that every transformer deeper than one layer undergoes an abrupt internal transition at roughly 2.5 to 5 billion training tokens.

Before that point: essentially no in-context learning.

After it: in-context learning simply exists.

The circuits responsible are called induction heads – attention patterns that spot “this happened before, here is what followed” and complete the pattern.

And here is the astonishing part.

This threshold barely depends on model size.

A 100-million-parameter model crosses it at roughly the same token count as a 500-billion-parameter one.

It is a time-based transition, not a size-based one.

You can see the bump in the loss curve with your naked eye.

It is the closest thing our field has to watching ice crystallize.

ChatGPT user:

• Percolation is the mathematics of sudden connection.
• Add random links to a network one at a time and for a long while you get only small isolated islands – then at one precise density, everything snaps together into a single connected whole.
• It is how forest fires spread, how epidemics take off, and how coffee finds its way through grounds.
• Capability emergence looks like exactly this: skills connecting into a network until, at a threshold, the whole thing lights up.

AI engineer:

• If you are pretraining or heavily continuing-pretraining a small model, budget past 5 billion tokens or you will not get few-shot capability at all, regardless of architecture.
• Below that line your model is fundamentally a different kind of object, and no amount of prompt engineering will rescue it.

Every time few-shot prompting works for you, you are using a circuit that formed in a single abrupt moment during that model’s training.

It was not designed.

Nobody programmed it.

It self-assembled.

---

The Numbers, One Last Time

Print this and pin it up:

• 20 tokens per parameter – the compute-optimal training ratio.
• 2.5-5 billion training tokens – where in-context learning switches on, at almost any model size.
• ~13 billion parameters – where multi-digit arithmetic appeared in 2022-era models.
• ~60-100 billion parameters – where chain-of-thought reasoning started helping instead of hurting.
• Critical pretraining loss – the real threshold behind all of the above. Everything else is a proxy.

The size numbers keep dropping.

The loss thresholds do not.

That is not a flaw in the theory.

That is the theory.

---

Conclusion: The Vast Avenues Ahead – and the Greatest Nonlinear System of All

So where does this leave us?

It leaves us with a field that has barely started.

Treating an LLM as a nonlinear dynamical system is not a cute reframing.

It is a research programme, and it opens doors in every direction.

Capability forecasting.

If skills are loss-threshold phenomena, we can predict what a model will do before we finish training it.

That is enormous – scientifically and commercially.

Safety.

Anthropic researchers warned in Predictability and Surprise in Large Generative Models that dangerous capabilities may appear abruptly rather than gradually.

Physics has spent a century learning to detect approaching phase transitions before they happen – divergent susceptibility, critical slowing down, growing correlation lengths.

Every one of those tools is now being pointed at training runs.

Early-warning systems for capability jumps are being built right now.

Better training.

Understanding grokking already produced Grokfast, an optimizer that deliberately accelerates the memorization-to-understanding transition.

That is theory paying rent within two years.

Architecture design.

If depth requires criticality, then designing for criticality directly – rather than stumbling into it through LayerNorm and residuals – is an obvious frontier.

Four avenues.

And they are only the ones visible from here.

But I want to close with the biggest one of all.

Because there is another nonlinear dynamical system that we have been trying to understand for far longer than we have had transformers.

It weighs about 1.4 kilograms.

It runs on twenty watts.

It sits behind your eyes as you read this sentence.

Your brain.

The human brain is the most complex nonlinear dynamical system we know of in the universe.

And – remarkably – real neuroscience has found the same signatures we have been discussing all article long.

Neural avalanches following power laws.

Cortical dynamics poised at criticality.

Phase transitions between states of consciousness.

Self-organized tuning to the edge of chaos.

The same mathematics.

The same fingerprints.

I am not claiming an LLM is a brain. It is not, and anyone who tells you otherwise is selling you something that is not rigorous science.

But I strongly believe we may have accidentally built the first nonlinear system complex enough to be interesting, simple enough to be studied, and – crucially – fully instrumented.

We can read every weight.

We can checkpoint every step.

We can rerun the whole thing with one variable changed.

You cannot do that with a brain.

So here is the possibility that keeps me up at night, in the best possible way.

What if the science we develop to understand emergence in language models becomes the science that finally cracks emergence in minds?

What if the tools we are building to detect a phase transition in a training run turn out to be the tools that explain how understanding arises in a child?

What if the deepest payoff of artificial intelligence is not the artificial part at all – but what it teaches us about the intelligence we already had?

Understanding.

Consciousness.

Meaning.

The image of God in the machinery of the mind.

I do not know if we will ever get there.

But I strongly believe the road runs through nonlinear dynamical systems, and I strongly believe it is the most fascinating road in science today – the road to understanding AI, and maybe even I, myself/itself.

The edge of chaos is where computation lives.

It may also be where meaning lives.

And we have only just started walking.

Watch this space!

---

Sources

1. Scaling Laws for Neural Language Models – Kaplan et al., 2020
2. Training Compute-Optimal Large Language Models (Chinchilla) – Hoffmann et al., 2022
3. Emergent Abilities of Large Language Models – Wei et al., 2022
4. Are Emergent Abilities a Mirage? – Schaeffer et al., 2023
5. Understanding Emergent Abilities from the Loss Perspective – Du et al., 2024
6. Grokking: Generalization Beyond Overfitting – Power et al., 2022
7. Progress Measures for Grokking via Mechanistic Interpretability – Nanda et al., 2023
8. Explaining Grokking Through Circuit Efficiency – Varma et al., 2023
9. Unified View of Grokking, Double Descent and Emergent Abilities – Huang et al., 2024
10. In-Context Learning and Induction Heads – Olsson et al., 2022
11. Critical Data Size of Language Models from a Grokking Perspective – Zhu et al., 2024
12. At the Edge of Chaos – Bertschinger & Natschläger, NeurIPS 2004
13. Exponential Expressivity Through Transient Chaos – Poole et al., 2016
14. Deep Information Propagation – Schoenholz et al., 2017
15. Attention Is Not All You Need: Rank Collapse – Dong et al., 2021
16. Predictability and Surprise in Large Generative Models – Ganguli et al., 2022
17. Grokfast: Accelerated Grokking by Amplifying Slow Gradients – Lee et al., 2024
18. A Review of Developmental Interpretability in LLMs – 2025

---

Every image in this article was generated by Nano Banana Pro.

The first draft of this article was written with Claude Opus 5.