← Architectures

Looped transformer inference

Recurrent-depth models apply the same block R times per output token. The dashed loop is where the extra "thinking" happens — in hidden state, not in visible tokens.

What the loop actually buys you

Effective depth becomes L × R — the number of layers in the block times the number of iterations — with zero extra parameters. You're spending compute along a new axis: not more parameters, not more tokens of chain-of-thought, but more forward passes per token.

The halt gate is the interesting piece. In an adaptive-halting variant, easy tokens exit after just a few iterations while hard tokens loop more. That makes per-token latency content-dependent instead of length-dependent. It's a different scaling knob from anything CoT models do.

Where the reasoning went

In a chain-of-thought model, intermediate reasoning is literally in the token stream — you can read it, log it, catch it going wrong. In a looped model, the same intermediate reasoning happens inside the residual stream between token emissions. From the outside you see input, then output. The middle is opaque.

That's the interpretability tension driving the current safety conversation about recurrent-depth architectures. I wrote about it in more depth in Reasoning off the page.