The Honest Scaling Story: Two Misses, Then Parity

The Pre-Registered Claim (H1): a weight-tied equilibrium LM reaches ≥95% of a parameter-matched explicit transformer's BLiMP at matched token budget. This was formally missed — twice — before it was effectively resolved.

Miss one (F18, 11M params, 3 seeds): solver-trained EqLM reached ratio 0.930 with a tight bootstrap CI [0.898, 0.949] — genuinely below the 95% bar, no rounding rescue. Miss two, worse (F20, 121M params, 3 seeds): the ratio fell to 0.787 [0.785, 0.788]. Adversarial review refuted our first mechanism (a binary solver failure at width) — the solver exhausted its 12-iteration budget at every scale; what grew with width was the price of that truncation.

The turn (F24): train the same weight-tied block unrolled — twelve explicit applications of the map with cross-entropy supervision at iterates z4, z8, z12 — and evaluate with the standard Anderson solver. Three seeds: BLiMP 0.662 / 0.697 / 0.672 → ratio 0.991, with one seed above its explicit baseline. Identical data, steps, batch, optimizer, and learning rate as the control (audited). Bonus: unrolled training is 2.1× faster than implicit-differentiation training — the equilibrium solve, not backprop, dominates EqLM's training cost.

Anytime property (F24 rider, pre-registered): the same checkpoint evaluated at solver budgets {4, 8, 12} scores gracefully (e.g. 0.601 / 0.674 / 0.697 on seed 43). One model, three inference budgets.

Honest costs: unrolled training stores its iterates, so it pays explicit-like activation memory during training (16.4 GB vs 7.8 GB); serving keeps the implicit model's O(1) depth-memory and warm-start decoding. And the solver still exits uncertified — quality and certification were solved in different arms (see the findings page, F24).