AIAny
Icon for item

Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs

Shows that Transformer LLMs output a superposition of next-token distributions when token embeddings from distinct text streams are linearly combined, finds this property is architectural but weakened by pretraining, and demonstrates it can be restored by lightweight fine-tuning and disentangled at decode time.

Introduction

Oops! Something went wrong

[next-mdx-remote-client] error compiling MDX: Unexpected character `0` (U+0030) before name, expected a character that can start a name, such as a letter, `$`, or `_` More information: https://mdxjs.com/docs/troubleshooting-mdx

Information

  • Websitearxiv.org
  • AuthorsPavel Tikhonov, Anton Korznikov, Matvey Mikhalchuk, Nikita Dragunov, Temurbek Rahmatullaev, Polina Druzhinina, Anton Razzhigaev, Ivan Oseledets, Elena Tutubalina
  • Published date2026/09/24

More Items

Edits an LLM agent's internal task state and action continuations to remove unsupported assumptions and noisy plans, improving downstream decisions instead of reconstructing high-entropy tool outputs. Combines Action Judge, State Revision, and EditAct and shows consistent gains across benchmarks and agent backbones.

Stores speaker-labeled verbatim messages alongside derived person- and group-level states to improve attribution and long-term memory in multi-party dialogues. Proposes a dual-track memory and a speaker-conditioned Writer trained with SpeakerLevenshtein and GRPO; yields consistent gains on multiple dialogue memory benchmarks.

Adds token-conditioned quantum residual branches to a frozen masked-diffusion language model: a lightweight hypernetwork emits continuous quantum-circuit coordinates per token, executes a shared sparse IQP-style circuit, and injects classically-expressible expectation readouts back into transformer blocks. Trains only the added branches, scaling to 16–64 qubits with analytic, linear-cost readouts.