AI Deploy2025
Turns OpenAI Whisper into a live streaming transcriber: audio flows in over WebSocket and text returns word-by-word instead of after full utterances. Adds SimulStreaming and LocalAgreement decoding, Silero VAD, and speaker diarization, all self-hosted.
AI Deploy2025
A ~5,000-line Python LLM inference engine that re-implements SGLang's serving optimizations — radix KV-cache reuse, chunked prefill, overlap scheduling, tensor parallelism — as a fully type-annotated reference instead of a black box.
AI Infra2025
Extends vLLM beyond text to serve omni-modal models — Qwen3-Omni, TTS like CosyVoice3, and diffusion image/video/audio generators — in one engine, adding the non-autoregressive Diffusion Transformer support the core project never targeted.
AI Audio2025
Delivers multilingual, on-device text-to-speech via ONNX Runtime with prebuilt ONNX assets and cross-platform SDKs (Python, Node, mobile); targets low-latency, privacy-preserving TTS with ready demos and 31-language support in v3.
MCP Server2025
Official MCP server for data.gouv.fr, France's national open-data portal: lets chatbots search datasets, query CSV/XLS via the Tabular API, and browse cataloged third-party APIs, all read-only over a public HTTP endpoint that needs no key.
AI Infra2025
Embeds into an app like SQLite, persisting to a local file with no server or separate process. Combines dense and sparse vectors, full-text search, and scalar filters in one hybrid query; C++ core with Python, Node, Go, Rust, and Dart bindings.
AI Deploy2026
Enables parallel speculative decoding by using a lightweight block-diffusion draft model to produce multi-token drafts for faster, high-quality generation. Integrates with vLLM, SGLang and Transformers backends and ships draft models on Hugging Face.
AI Deploy2026
Orchestrates low-latency, multi-stage pipelines for omni and multimodal models by running each stage with its own scheduler and using zero-copy shared memory for tensor transfer. Emphasizes per-stage bottleneck tuning and OpenAI-compatible streaming endpoints, suitable for TTS and multimodal serving.
AI API2026
Converts DeepSeek protocol calls into OpenAI/Claude/Gemini-compatible APIs with a Go backend and React admin UI. Offers account pooling, protocol adapters, tool-call translation, PoW, and multiple deployment modes (Docker, Vercel, standalone).
AI API2026
A transparent proxy that lets Claude Code CLI and the VSCode extension run without an Anthropic API key by routing requests to NVIDIA NIM, OpenRouter, LM Studio, or llama.cpp; supports per-model mapping, thinking-token parsing, and messaging integration.
AI API2026
Unified API proxy and protocol gateway that translates and routes requests to Claude, OpenAI Chat/Images/Codex, and Gemini. Offers channel orchestration, multi-key rotation, failover, model routing, and a built-in web admin UI for consolidating multiple model providers behind a single endpoint.