Menu
Article

Reasoning Got Adaptive. Understanding Didn't.

Vanis Lim

Vanis Lim

• 8 min read

Notes on where I think the architecture goes next — September 2026

This started from a dumb question.

I was looking at how neural nets get drawn — layers stacked like sheets, one on top of the next — and I thought: why is it flat? A brain is a three-dimensional thing, wired in every direction. If inference were shaped more like that, wouldn't it be more efficient? Wasn't that the unlock everyone was missing?

It's a naive question. But I've learned that if I follow a naive question honestly instead of dropping it because it's embarrassing, it usually walks me somewhere I wouldn't have reached on purpose. This one did.

The dimension idea fell apart first. The units inside a model don't sit anywhere — they have no coordinates, you can shuffle them and the network is identical. "2D" is the whiteboard, not the computation. So I moved the goalpost to depth: surely deeper is the thing, more layers to think through. That fell apart harder, because the numbers say the opposite — DeepSeek-V3 runs 61 layers and beats GPT-3's 96. Depth roughly stopped scaling years ago; the growth went into width, and then into sparse width. So it's not dimensions and it's not depth.

What was actually bugging me took a few more turns to surface. It's this.

we only made half of it adaptive

A model's forward pass does a fixed amount of computation. The prompt goes in, it runs through every layer once, and out the other end is the internal state the model reasons from. Trivial prompt or the hardest thing it'll see all day — same layers, same cost, every time.

Chain-of-thought didn't change that. What CoT made adaptive is everything after the first token: the model spends more tokens on harder problems, recycling its own output back into itself to climb toward a better output. That's real, and it's most of why reasoning models are good. But notice where the adaptive compute lives — entirely in the output phase, after the model has already committed to a first token.

The phase it never made adaptive is the one before that. The understanding phase. The window before time-to-first-token, where the model reads your input and turns it into something it can reason over. That's still a single fixed pass. The model has no way to spend longer understanding a hard input before it starts talking — which is exactly the phase where a person spends the most time on something genuinely difficult. You sit with the problem before you open your mouth. The model can't. Its TTFT is a constant set by the architecture, not by the difficulty of what you asked.

That's the asymmetry that finally felt like the real thing. We made the reasoning adaptive and left the understanding fixed. CoT improves the talking, not the thinking-before-talking. Half a system.

And once I saw it that way, the two things I'd been circling both turned into answers to the same gap.

idea one: it's not either/or

The obvious reaction is "so replace chain-of-thought with something that thinks internally instead." That's the wrong instinct, and it took me a while to see why.

Watch how a person handles something hard. There's a part where you take it in — sit with it, turn it over, let it settle, before you've said a word. And there's a part where you reason out loud, work the steps, each thing you say feeding the next. Two different activities, and one doesn't replace the other. You digest and you reason.

Current models basically only have the second, and only recently. CoT is the reasoning-out-loud — it literally thinks in output space, folding its own drafts back into better drafts. It works, but it's an awkward shape: every thought has to be squeezed through tokens first, and it doesn't touch TTFT at all. The digesting — the adaptive understanding before the first token — is just missing.

That's the half worth building, and it's a real line of work now, not a hypothetical. Geiping's recurrent-depth setup iterates a block in latent space and lets each token unroll to whatever depth it needs — hard tokens loop more, easy ones exit early, all of it before anything is emitted. Coconut lets a model reason in continuous latent space instead of forcing every thought through tokens. This is adaptive test-time compute spent before first token, in the model's own representation, which is exactly the phase CoT leaves fixed.

And the point that matters most to me: it doesn't compete with CoT, it sits underneath it. Adaptive latent digestion before first token, adaptive token reasoning after. Two phases, two budgets — and they multiply. A genuinely hard problem might want both deep latent iteration and a long chain. So the real design question was never which one; it's how you split the budget between thinking-before-you-speak and thinking-while-you-speak, per problem, by difficulty. Nothing in the architecture stops you having both. We built the second half first because that's where the benchmarks and the money were. The first half is the more human of the two, and it's the one nobody's spending on.

idea two: stop making it think in one direction

Here's the other thing that was nagging at me, and I think it's the deeper one.

Look at how a big model actually routes today. In a mixture-of-experts model the router fires at every layer — in something like DeepSeek-V3 it picks 8 experts out of 256 at each of 61 layers, sums them, moves on. Two things are fixed there that don't need to be. It's always 8, regardless of the token — the throwaway word and the one hard token the whole prompt hinges on get the same expert budget. And it only ever flows one way: the experts at layer 40 work on whatever layer 39 handed them. They can revise it, but they're always framed by it. Information goes up the stack and never back down.

Both of those are what I'd change, and they turn out to be the same change.

Take the one-directional part first, because it's the one that nags. The best picture I have is an appeals court: the higher court can overturn the lower one, but it's still working from the record the lower court produced, inside the framing it set. Review, one direction, all the way up. That's not how you actually work a hard problem. What you want is a panel — the experts arguing back and forth, lateral peering links between them and not just the vertical flow up the layers, iterating until the room converges, with no fixed number of rounds. Easy input settles fast; hard input keeps the relevant experts talking to each other longer. Nobody's permanently stuck as the early judge whose framing everyone downstream inherits, because it isn't a line, it's a loop.

That shape has a name — equilibrium or attractor models, where instead of running a fixed stack front to back you let the system settle until it stops changing. Deep equilibrium models have done this since about 2019; the recent looped and universal-transformer work is the same instinct, turning depth into iteration you can dial. The one-directionality that bothered me isn't fundamental. It's a property of the feedforward shape, and feedforward was never the only option — just the one we picked.

And the fixed-budget part folds straight into the loop. Once you're iterating anyway, let the number of active experts and the number of rounds be dynamic — set by the token's context, not by a constant. Concentrate the budget where the problem actually is instead of splitting it evenly across experts that don't matter for this input. Mixture-of-depths already does a slice of this — it lets tokens dynamically skip layers, which is per-token compute allocation in a smaller form. Push it all the way: a global expert budget that flexes with difficulty, spent on the experts that matter and the rounds they need, with expert-to-expert peering so they can correct each other instead of only inheriting from below. That's the dynamic resource allocation the current architecture can't do, because the current architecture is a straight line with a fixed width.

Then the part I got most excited about, because it kills the obvious objection. The reason everything is feedforward is that feedforward is clean to train — the straight-line, one-directional shape (a DAG) is exactly what makes backprop scale to these sizes. Loops are lovely to think in and historically miserable to train. But you don't have to use the same shape for both. Train straight, think in loops. Take a model trained the normal, cheap, stable feedforward way, and let it deliberate — iterate, route dynamically, settle — at inference. People are already doing the conversion, retrofitting recurrence into ordinary pretrained models as a post-training step. Feedforward to build it, deliberative to run it. That transition — DAG to train, panel to think — is the thing I'd bet on.

why I think this is the direction

Put the two together and you get something current systems can't do. Spend a variable amount of latent compute understanding the input before the first token, scaled to how hard it is. Then reason in tokens after, also scaled. Route dynamically and bidirectionally the whole way through, concentrating experts and iterations where the problem lives instead of spending a flat budget in a straight line. And train the whole thing feedforward so it's still buildable. That's a lot closer to how anything that actually understands things works — you take the time you need, most of it before you speak, and you spend it where it matters.

Is it built and proven at frontier scale? No. There's real work left — training the loop, keeping it stable, making it settle cleanly, splitting the budget well. But every problem I can point to is an engineering problem, and I've watched too many "that's fundamentally impossible" things turn out to just need the right implementation to bet against this class of them. Impossible, most of the time, only means nobody's found the clever version yet. The gap here isn't a law of physics. It's work.

I'm mostly writing this down so future-me can check it. It's not a paper and I haven't solved anything — it's just the honest shape of where I think this goes, traced from a dumb question about why the diagram is flat to something I've talked myself into being fairly sure of. If I'm wrong, I'd like to know exactly where. And if I'm right, I want to have said it before it was obvious.

the work this leans on

Share