Hybrid Reasoning Agents: LLM Agents That Think in Latent Space and Act Through Text
Hybrid Reasoning Agents, an October 2026 arXiv preprint by Jiaru Zou, James Zou, and colleagues, trains models to deliberate in latent space, feeding hidden states back without writing tokens, and switch to text for tool calls and answers. The authors report fewer tokens and faster inference. The cost is a reasoning trace you can no longer read directly.
The paper
"Hybrid Reasoning Agents" is an arXiv preprint (2610.18861), submitted on October 13, 2026, by Jiaru Zou, Yunzhe Qi, Sirui Chen, Federico Bianchi, Shang Zhu, Pan Lu, Jingrui He, and James Zou. The paper lists affiliations with Stanford University, the University of Illinois Urbana-Champaign, and Together AI. It is 45 pages with 12 figures and lists no conference venue, so treat it as unreviewed work.
The idea is in the subtitle that circulated on Hacker News: agents that think in latent space and act through text. Instead of writing every reasoning step as visible tokens, the model can switch into a mode where it feeds its own hidden state back as the next input, then switch back to text when it needs to call a tool, read an observation, or answer.
Text chain of thought versus latent reasoning
In ordinary chain of thought, each reasoning step is sampled as a token. Sampling collapses a rich internal state, a high-dimensional vector, into one discrete word, and the next step starts from that word. The reasoning is readable, but every step costs a generated token and loses whatever the chosen word did not capture.
Latent reasoning skips the collapse. The model's last-layer hidden state is mapped back into the input embedding space and fed forward directly, so the next step continues from the full vector. Earlier work such as Coconut, SoftCoT, and SoftThinking, all baselines in this paper, explored this for single-shot reasoning. The open question was agents, where the model must also emit exact text: tool names, arguments, search queries, final answers.
How HybridAgents work
The paper's design has four parts, as the authors describe them.
Mode switching with special tokens
In text mode the model can emit a <latent> token to enter latent mode. It leaves when the nearest token to its latent state decodes to </latent>. The policy for when to switch is learned, not hard-coded.
HybridLink
A lightweight residual adapter that maps the last-layer hidden state back into the input embedding space. The paper puts it at roughly 10 million parameters for a 4B model.
Continuous working memory
The key-value cache entries from text, latent thoughts, and tool observations persist across turns, so the agent carries its full hybrid context forward without re-tokenising it.
HybridSFT and HybridRL
Supervised fine-tuning stabilises tool use inside hybrid generation, and reinforcement learning teaches the model when to reason latently and when to act through text.
What the authors report
The models are Qwen3-Base at 4B, 8B, and 14B. Evaluation covers 11 benchmarks across math, science, medicine, search, and agentic tasks, including AIME, MATH500, GPQA-Diamond, MedQA, GAIA, Humanity's Last Exam, and WebWalker. Baselines include text-trained agents, latent reasoning methods, frontier LLM agents, and frameworks such as ReAct and Search-o1.
The abstract reports a 10.4 percent average improvement over frontier LLM agents and agentic frameworks, 35.5 to 54.7 percent fewer tokens, 1.8 to 2.6 times faster inference, and 2.7 to 3.7 times lower cost. These are the authors' numbers on their own evaluation. The appendices note that the cost estimates exclude prompt prefill, context-dependent attention, and external tool execution, which in real agents is often where much of the time and spend goes.
The trade-offs for agent builders
The efficiency argument is plausible: fewer generated tokens for internal deliberation, while tool calls and answers stay exact. The costs are elsewhere.
You lose the readable trace
Text chain of thought is the main way teams debug agents and monitor them for unsafe plans. Latent steps are vectors. The paper offers nearest-token projection as pseudo-text and reports readability scores, but its own appendix calls this a geometric proxy. The authors deliberately keep decision-critical steps in text for auditing. Anyone adopting this should decide which decisions must stay in text and enforce it.
Errors can drift silently
The appendices note that latent spans lack a discrete projection, so errors can compound over tens to hundreds of steps and erode coherence. A text step that goes wrong is visible; a latent one is not.
It needs open weights and custom inference
HybridLink, latent feedback, and persistent KV caches require control of the model and the serving stack. Closed APIs that bill per text token do not expose any of this. For now, this is a technique for teams running their own models.
Working memory is not long-term memory
A KV cache is precise within a session but large, tied to one model version, and gone when the session ends. Knowledge an agent needs next week, or that another agent needs, still belongs in an external store it can retrieve from. Latent reasoning reduces deliberation tokens; retrieval reduces the reference tokens an agent rereads, which on repeated work is where RDK's 80 to 90 percent reduction applies. They solve different halves of the cost.
Who should pay attention
Teams training or fine-tuning their own agent models, especially for long multi-step tasks where deliberation dominates token count, should read the paper and watch for released code. Teams on hosted frontier APIs should take a narrower lesson: reasoning tokens are a real cost line, and moving routine deliberation out of visible text trades auditability for efficiency. That is a choice to make deliberately, step by step, not by default.
Frequently asked questions
- Who wrote the Hybrid Reasoning Agents paper?
- Jiaru Zou, Yunzhe Qi, Sirui Chen, Federico Bianchi, Shang Zhu, Pan Lu, Jingrui He, and James Zou, with affiliations listed at Stanford University, UIUC, and Together AI. It was posted to arXiv as 2610.18861 on October 13, 2026. No conference venue is listed, so it has not been confirmed as peer reviewed.
- What is latent space reasoning in LLMs?
- Latent reasoning lets a model continue thinking from its internal hidden state instead of first converting each step into a written token. The hidden vector is fed back as the next input. It can carry more information per step and generate fewer tokens, but the reasoning is no longer directly readable the way text chain of thought is.
- Is latent reasoning safer or less safe than chain of thought?
- It is harder to monitor. Text chain of thought can be read, logged, and checked for bad plans. Latent steps are vectors that can only be approximated as text, and the paper itself describes its readability measure as a geometric proxy. Keeping consequential decisions, such as tool calls and approvals, in text preserves an auditable record.
- Can I use hybrid reasoning with a hosted API like Claude or GPT?
- Not as described in the paper. The method needs a trained adapter, control over how hidden states are fed back, and persistent key-value caches, which require open weights and your own inference stack. Hosted APIs expose text tokens only. The practical lessons for API users are to budget reasoning tokens and decide which steps must stay visible.