Most long-context systems still store or transmit summaries as text or images that must be decoded back into tokens before an LLM can use them. LatentPress flips that pipeline: instead of reconstructing human-readable context, it writes compact continuous tokens that a frozen decoder ingests directly, trading interpretable format for speed and model-aligned compression.
Key Findings
- Direct soft-token interface: A reader-matched writer maps multi-turn conversational histories and long documents to short continuous token sequences that are concatenated into a frozen decoder's input-embedding layer, eliminating runtime text reconstruction.
- Strong compression with a tiny trainable footprint: The writer trains only a small adapter (≈4.2M–26.2M parameters, ~0.1% of the decoder) and achieves 4–16× compression while preserving or improving downstream accuracy in many settings.
- Empirical gains and efficiency: On LongMemEval, LatentPress reached 0.504 accuracy at 7.70× compression (versus 0.490 with uncompressed evidence). Encoding is fast (~43 ms per conversation) and reading from the compressed prefix is 5–9× faster than reading raw context or cached OCR.
Who it's for and trade-offs
Great fit if you need machine-facing, high-throughput long-context storage or retrieval for frozen LLM decoders—e.g., scalable memory systems, long-document QA backends, or cross-session agents where latency and storage matter. Look elsewhere if human-readable archives, explainable reconstructed text, or full generalization across arbitrary downstream decoders are required—LatentPress favors direct machine consumption and depends on reader/decoder compatibility.
Where it fits
LatentPress sits between text-summary and encoder-decoder latent compressors: it keeps the downstream decoder frozen (reducing fine-tuning cost) while using soft tokens as a portable, fast context interface applicable beyond just text and vision.