AI Deploy2026
Enables parallel speculative decoding by using a lightweight block-diffusion draft model to produce multi-token drafts for faster, high-quality generation. Integrates with vLLM, SGLang and Transformers backends and ships draft models on Hugging Face.
AI Deploy2026
Orchestrates low-latency, multi-stage pipelines for omni and multimodal models by running each stage with its own scheduler and using zero-copy shared memory for tensor transfer. Emphasizes per-stage bottleneck tuning and OpenAI-compatible streaming endpoints, suitable for TTS and multimodal serving.
AI API2026
Converts DeepSeek protocol calls into OpenAI/Claude/Gemini-compatible APIs with a Go backend and React admin UI. Offers account pooling, protocol adapters, tool-call translation, PoW, and multiple deployment modes (Docker, Vercel, standalone).
AI API2026
A transparent proxy that lets Claude Code CLI and the VSCode extension run without an Anthropic API key by routing requests to NVIDIA NIM, OpenRouter, LM Studio, or llama.cpp; supports per-model mapping, thinking-token parsing, and messaging integration.
AI API2026
Unified API proxy and protocol gateway that translates and routes requests to Claude, OpenAI Chat/Images/Codex, and Gemini. Offers channel orchestration, multi-key rotation, failover, model routing, and a built-in web admin UI for consolidating multiple model providers behind a single endpoint.
AI Infra2026
Self-hosted coding assistant that runs frozen local LLMs with constraint-driven planning, energy-based verification, and self-verified repair to produce verified code. Emphasizes offline inference (no cloud), Docker/bare-metal deployment, and requires a 16GB+ GPU.
AI Image2026
Multimodal OCR and document-understanding toolkit for recognizing complex layouts, tables, formulas and code. Uses Multi-Token Prediction and stable RL for better training; ships as a 0.9B-parameter model with a Python SDK and deployment guides for vLLM, SGLang and Ollama.
AI Infra2026
Runs untrusted code (LLM outputs, plugins, and third‑party tools) inside cross‑platform, policy‑driven sandboxes. Provides a unified JSON schema and a TypeScript SDK that sit on multiple containment backends (process sandboxes, LXC/Bubblewrap, microVMs). Early preview with known permissive profiles — not yet a security boundary.
AI Deploy2026
Runs a local-first, full AI stack—LLM inference, chat UI, voice, agents, workflows, RAG, and image generation—deployable with one command. Auto-detects hardware and bootstraps a small model for instant chat while larger models download; supports Linux, Windows, macOS and optional cloud/hybrid modes.
AI Deploy2026
Local LLM inference server for Apple Silicon that exposes an OpenAI-compatible API and a macOS menubar app. Uses continuous batching and a two-tier KV cache (RAM + SSD in safetensors) to persist context across restarts, enabling practical multi-model serving and fast local coding workflows.
AI Infra2026
Provides a reliability layer for self-hosted LLM tool-calling and multi-step agent workflows. Adds guardrails — rescue parsing, response validation, retry nudges, and a synthetic respond tool — and ships a Drop-in OpenAI-compatible proxy plus a WorkflowRunner for structured loops.