Why this matters
Searchable embedding corpora are growing fast and memory is often the limiting cost. A 10M-document, 1536-d embedding set costs tens of GB as float32; by using a data-oblivious, analytically derived quantizer and per-vector corrections, turbovec compresses those vectors into a few bits each while preserving retrieval quality and enabling much larger local indexes.
What Sets It Apart
- Online, no-train ingest: vectors are quantized and indexed on add; there is no separate training or rebuild step as the corpus grows, which simplifies streaming and continuous-collection workflows.
- TurboQuant-based compression + length renormalization: applies a random rotation and precomputed Lloyd–Max scalar quantizers (TurboQuant/TQ+) then stores a per-vector scalar to correct inner-product bias, improving recall especially at low bit widths.
- High-performance SIMD kernels: hand-tuned NEON and AVX-512BW implementations deliver strong throughput — benchmarks in the repo report ~12–20% speed gains over FAISS FastScan on ARM and parity or small wins on x86 for many configs.
- Filtered search inside the kernel: allowlists / slot bitmasks are honoured at search time so selective filtering doesn't force over-fetch or large recall loss, making hybrid pipelines (SQL/BM25 -> dense rerank) efficient.
- Local / air-gapped friendly: pure local runtime and file-backed indexes (Rust core, Python bindings) make it suitable for privacy-sensitive RAG stacks.
Who It's For — trade-offs and guidance
Great fit if you need to host large embedding indexes locally with tight memory and latency constraints (e.g., multi-million vectors on a single machine), want streaming/online ingest without retraining, and prefer a compact on-disk format for snapshotting or air-gapped deployments.
Look elsewhere if you require production-managed vector services, expect to run at extremely high recall at the lowest possible bit rates in every dataset (very-low-dim regimes can be harder for scalar quantizers), or depend exclusively on hardware without AVX2/NEON fallbacks for peak throughput. Also, being a local Rust-based library, operational integration (deployment, monitoring) is your responsibility rather than a hosted offering.
Where it fits in a stack
turbovec is best used as the dense vector index in a hybrid retrieval pipeline (candidate generation by DB/BM25 → dense rerank). It competes with FAISS-style PQ indexes on the memory/throughput/recall frontier but prioritizes online ingest and predictable, analytic quantization (no codebook training). The project provides integrations for common retriever frameworks so it can be dropped into LangChain, LlamaIndex, or Haystack pipelines.
