AI Agent2025
Provides a visual, low-code environment to build, debug, and deploy AI agents—integrates model services (OpenAI, Volcengine), RAG, plugins, workflows, and a Chat SDK for embedding agents into apps.
AI Infra2025
Stores a pruned proximity graph instead of all embeddings, recomputing vectors on demand at query time. A 60M-doc index takes 6GB, not 201GB (97% less), at comparable recall. Powers private local RAG over files, mail, chat, and browser history.
MCP Server2025
Centralized enterprise platform to manage org-wide MCP servers with a private MCP registry, security guardrails, cost controls, and observability. Offers a Kubernetes-native orchestrator, built-in RAG knowledge base, security sub-agents, and tools for governed AI adoption.
AI Client2025
Coordinates specialized AI agents — developer, browser, document, multimodal — running in parallel on your desktop to automate multi-step work. Runs fully local via Ollama, vLLM, or LM Studio, with built-in MCP tools and human-in-the-loop checkpoints.
AI Deploy2025
Turns OpenAI Whisper into a live streaming transcriber: audio flows in over WebSocket and text returns word-by-word instead of after full utterances. Adds SimulStreaming and LocalAgreement decoding, Silero VAD, and speaker diarization, all self-hosted.
AI Deploy2025
A ~5,000-line Python LLM inference engine that re-implements SGLang's serving optimizations — radix KV-cache reuse, chunked prefill, overlap scheduling, tensor parallelism — as a fully type-annotated reference instead of a black box.
AI Audio2025
Build and self-host production voice agents with a drag-and-drop workflow builder, real-time telephony integration, and pluggable LLM/STT/TTS backends. Docker-first with an optional managed cloud offering for teams that want faster onboarding.
AI Infra2025
Extends vLLM beyond text to serve omni-modal models — Qwen3-Omni, TTS like CosyVoice3, and diffusion image/video/audio generators — in one engine, adding the non-autoregressive Diffusion Transformer support the core project never targeted.
AI Video2025
Runs text-to-video, image-to-video, text-to-image, and image editing inference with acceleration, offloading, quantization, and distributed execution for large visual generation models.
AI Deploy2026
Enables parallel speculative decoding by using a lightweight block-diffusion draft model to produce multi-token drafts for faster, high-quality generation. Integrates with vLLM, SGLang and Transformers backends and ships draft models on Hugging Face.
AI Deploy2026
Orchestrates low-latency, multi-stage pipelines for omni and multimodal models by running each stage with its own scheduler and using zero-copy shared memory for tensor transfer. Emphasizes per-stage bottleneck tuning and OpenAI-compatible streaming endpoints, suitable for TTS and multimodal serving.