AI Train2024
Accelerates video generation with a unified framework for inference, finetuning, LoRA, distillation, sparse attention, and distributed execution for research and demos.
AI Agent2024
React components for building LLM chat and agent interfaces: message bubbles, prompt sets, conversation lists, and sender inputs under a RICH interaction paradigm, plus a streaming Markdown renderer and hooks for wiring UI to model data streams.
MLOps2024
Runs reproducible evaluations of large language models through a Python API with built-in solvers, scorers, and model-graded grading. Ships 200+ ready-to-run evals spanning capability and safety testing, and connects to most major model providers.
AI Client2024
Web-based resume editor with real-time preview, custom themes, dark mode, auto-save and PDF export, plus built-in AI-assisted writing and a custom model for polishing content. Open-source under Apache-2.0 but requires a commercial license for paid/enterprise use.
AI Infra2024
Stores and reuses LLM key-value caches across GPU, CPU, disk, and remote backends so vLLM and SGLang skip recomputing repeated context. Non-prefix reuse (CacheBlend) and PD disaggregation cut time-to-first-token for long-context and RAG serving.
AI Train2024
A community speedrun to train a 124M GPT as fast as possible on 8 H100s, all chasing a fixed 3.28 FineWeb loss. Successive records cut the run from llm.c's 45 minutes to under 1.4, mostly via the new Muon optimizer rather than more hardware.
AI API2024
Ingests documents, images, audio, video and web pages and converts them into structured, LLM-friendly markdown and parsed data. Runs locally (fits on a T4 GPU), supports ~20 file types, offers OCR, transcription, table extraction and a Gradio UI; deployable via Docker/Skypilot. Licensed under GPL-3.0; some model weights carry cc-by-nc-sa restrictions for commercial use.
AI Infra2024
Cloud-native control plane that scales vLLM on Kubernetes, adding the routing, autoscaling, and fault tolerance single-instance serving lacks. Brings high-density LoRA management, an LLM gateway, distributed KV cache reuse, and SLO-aware GPU serving.
AI Client2024
Register React components with Zod schemas so an LLM agent can select, fill, and stream their props from user requests, turning chat into live interactive UI. Works with OpenAI, Anthropic, Gemini, and Mistral, plus MCP servers like Linear.
MCP Server2024
Continuously records your screen and audio 24/7 to a local, searchable timeline you can query in natural language. Stores screenshots with accessibility data in SQLite, and a plugin system runs scheduled AI agents on what it captures.
AI Agent2024
Builds real-time multimodal conversational AI agents with voice-assistant examples, VAD, turn detection, RTC/WebSocket transport, avatars, transcription, and edge-device demos.