Persistent Memory for Continuous Latent Reasoning. We find that latent-reasoning LLMs forget what they just figured out โ and fix it with three tiny learned gates and a memory stream that costs just 1.41% extra parameters.
In latent reasoning (CoCoNuT), each reasoning pass overwrites the previous hidden state. Facts computed early are lost by the time the answer is generated โ 87% of pass-1 information is gone by pass 6.
AGCLR adds a persistent concept stream โ a residual memory across passes โ controlled by read, forget, and write gates. Like an LSTM, but gating memory across reasoning passes, not timesteps.
+2.6% on GSM8K, +3.6% EM on HotpotQA, +4.0% on ProsQA over vanilla CoCoNuT โ and the gap grows with reasoning depth, exactly where the bottleneck bites hardest.
Latent reasoning feeds the model's hidden state back as the next input โ powerful, but each pass overwrites the last. Drag the slider to simulate reasoning passes and watch what happens to the facts. Values are real cosine similarities from the paper (Fig. 3).
At every latent token position, a persistent vector ct (the concept stream) rides alongside the hidden state ht. Three sigmoid gates decide what gets remembered, retrieved, and pruned. Click a gate to learn what it does.
Each gate is a learned sigmoid over the layer-normed hidden state โ e.g. rt = ฯ(Wr ฤฅt). Together they cost only 1.41% extra parameters on GPT-2.
ฤฅโ = LayerNorm(hโ)normalize the current thoughtrโ, fโ, wโ = ฯ(Wยทฤฅโ)three gates, each โ [0,1]แตhโฒโ = (1โfโ)โhโ + rโโcโโโprune noise, pull in memorycโ = LayerNorm(cโโโ + wโโhโฒโ)commit to the streamSlide the three gates and watch how the next hidden state hโฒt gets mixed, and how fast the concept stream fills up. The presets are the paper's actual learned initializations (Appendix A.3).
All methods use the same GPT-2 (117M) base. Pick a dataset โ bars animate to the paper's Table 1. Hover a bar for details.
Cosine similarity of each pass's hidden state to pass 1 (100 validation samples, epoch 15). Vanilla decays monotonically to 0.126 (โ87% information loss); AGCLR stabilizes at 0.216 โ retaining 71% more information at final generation.
On ProsQA, vanilla CoCoNuT peaks at 95% at curriculum stage 5, then degrades to 92% at stage 6 (fully latent). AGCLR keeps climbing to 96% โ the concept bottleneck, resolved.
The pattern is consistent: AGCLR and vanilla perform similarly at early stages, then the gap compounds as more reasoning goes latent โ exactly where a persistent memory should matter.
Probing the concept stream against the vocabulary (logit-lens style) shows it stores answer-relevant entities at 0.6โ0.8 cosine similarity. Three cases where AGCLR wins and vanilla hallucinates โ click a card to flip it.
The stream keeps the PennsylvaniaโWilliam Penn binding; vanilla loses it across passes and hallucinates a plausible historical figure.
Preserved temporal context stops the drift toward the statistically more common "World War II".
Both the base entity and abstraction markers survive, enabling the second hop from state โ country. Vanilla stops at hop one.
Is it just extra weights? No. Freezing the write gate after pass 2 costs only โ0.8% EM โ the stream is persistent storage, not a scratchpad: write early, read often. And every gate earns its keep (HotpotQA EM).
Biggest drop from removing forget (โ5.6): without pruning, the stream gets polluted with noise. All three gates are indispensable.
Single-seed runs on GPT-2 124M across three benchmarks; scaling to 1B+ models, multi-seed variance estimates, and broader benchmarks (MuSiQue, 2WikiMultihopQA, StrategyQA) are future work. The architecture itself is parameter-agnostic. Total training: ~8 hours on a single A10 GPU.
@inproceedings{farhan2026agclr,
title = {Why Limit the Residual Stream to Layers and Not Tokens?
Persistent Memory for Continuous Latent Reasoning},
author = {Farhan, Mujtaba and Chaudhary, Maheep},
booktitle = {Proceedings of the 43rd International Conference
on Machine Learning (ICML)},
year = {2026},
publisher = {PMLR}
}