Most "chat with your docs" tools quietly ship your files to a cloud API, and the privacy promise lives in a terms-of-service page you never read. This project inverts that default: the entire retrieval and generation loop runs on hardware you control, so the interesting question stops being "is my data safe?" and becomes "how good can local RAG actually get?" Its answer treats retrieval quality, not the model, as the thing worth engineering.
What Sets It Apart
- Hybrid retrieval blends dense semantic search, keyword matching, and Late Chunking, so long documents stay coherent and answers cite a whole relevant passage instead of a fragment torn from context.
- A query router decides per question whether to run full RAG or answer straight from the LLM, cutting latency and noise on questions that need no document lookup.
- It layers in reliability features most local stacks skip: query decomposition for multi-part questions, semantic caching to avoid recomputing similar queries, contextual enrichment, and an answer-verification pass that checks responses against retrieved evidence.
- Model-agnostic by design, swapping between Ollama-hosted and HuggingFace models without rewiring the pipeline.
Who It's For
Great fit if you handle sensitive material such as legal, medical, or internal research documents, or simply want a private knowledge base you can run on a workstation and tinker with; the modular, mostly-Python codebase rewards customization. Look elsewhere if you want a turnkey hosted product or need to serve many concurrent users at scale, since this is a self-hosted system whose answer quality and speed track the local models and GPU you point it at.
