AI Infra2023
Provides a PyTorch-native platform for experimenting with and scaling generative AI training, including composable parallelism, checkpointing, float8, logging, and Llama recipes.
AI Infra2023
Triton kernels and PyTorch layers for linear-attention, state-space, and sparse-attention token mixers (GLA, RWKV, Mamba2, GSA) as drop-in replacements for multihead attention. Runs on NVIDIA, AMD, and Intel GPUs with Hugging Face support.
AI Infra2023
Traces how Transformer LLMs route information from input to output, attributing each block's effect to individual attention heads and feed-forward neurons. Click any edge to see what a head promotes or suppresses in vocabulary space.
AI Audio2023
Builds real-time voice and multimodal AI agents as composable streaming pipelines. Vendor-neutral: swap among 20+ STT, 20+ LLM and 30+ TTS providers over WebRTC or WebSockets, and compose multi-agent systems with handoff and parallel workers.
AI Agent2024
Organizes reusable AI prompts as Markdown 'Patterns' you run from the CLI — summarize a video, extract claims, rate content. Switch among 20+ providers (OpenAI, Claude, Gemini, Ollama) and reach them via CLI, web UI, or REST API.
Re-derives LLM scaling laws, tracing prior disagreements to how compute budget was modeled, then trains 7B and 67B models on 2T tokens. The 67B model beats LLaMA-2 70B on code, math, and reasoning; its chat variant tops GPT-3.5 on open-ended evals.
AI Infra2024
Serves large language and multimodal models with low latency and high throughput using RadixAttention, continuous batching, structured outputs, parallelism, quantization, and broad accelerator support.
AI Audio2024
Lets AI agents place and answer business phone calls, holding spoken conversations to collect structured data, answer questions, and escalate to humans. Built on Azure Communication Services and Azure OpenAI, with RAG over your own documents.
Reworks the MoE layer to push each expert toward a narrow specialty: split experts into many finer ones and activate more per token, plus reserve a few always-on shared experts for common knowledge. A 2B model matches GShard 2.9B; at 16B it rivals LLaMA2 7B on ~40% of the compute.
AI Agent2024
Python framework for building and serving LLM agents in production: a unified event bus for real-time frontends and human-in-the-loop, fine-grained tool permissions, multi-tenant serving, and tool/code execution sandboxed via Docker or E2B.
AI Agent2024
Gives developers low-level primitives for building stateful single-agent, multi-agent, and graph-based control flows, with built-in human-in-the-loop checkpoints, persistent cross-session memory, and token-level streaming.
AI Client2024
GPU-accelerated code editor written in Rust, organized like a game engine to render its UI via shaders. Includes native agentic coding over the open Agent Client Protocol, multiplayer editing, LSP/DAP, and an open edit-prediction model.