AI API2024
Ingests documents, images, audio, video and web pages and converts them into structured, LLM-friendly markdown and parsed data. Runs locally (fits on a T4 GPU), supports ~20 file types, offers OCR, transcription, table extraction and a Gradio UI; deployable via Docker/Skypilot. Licensed under GPL-3.0; some model weights carry cc-by-nc-sa restrictions for commercial use.
AI Infra2024
Cloud-native control plane that scales vLLM on Kubernetes, adding the routing, autoscaling, and fault tolerance single-instance serving lacks. Brings high-density LoRA management, an LLM gateway, distributed KV cache reuse, and SLO-aware GPU serving.
AI Client2024
Register React components with Zod schemas so an LLM agent can select, fill, and stream their props from user requests, turning chat into live interactive UI. Works with OpenAI, Anthropic, Gemini, and Mistral, plus MCP servers like Linear.
A 236B-parameter Mixture-of-Experts LLM that activates only 21B parameters per token, cutting training cost 42.5% versus a dense 67B model and shrinking the KV cache 93.3% via Multi-head Latent Attention, with 128K context.
AI Agent2024
Provides a Python framework for building generative-AI agents and workflows with Pydantic-style type safety and composable capabilities. Model-agnostic provider support, built-in observability, human-in-the-loop tool approval, and durable execution for production use cases.
AI Infra2024
Connects multiple Macs and Linux machines into one cluster to run models too large for any single machine. Auto-discovers peers, shards a model across them via tensor parallelism, and exposes OpenAI-, Claude-, and Ollama-compatible APIs.
AI Client2024
Visually edit Next.js + Tailwind projects in the browser like Figma, with every change written straight back to your real React code. Pairs a DOM-level visual canvas with AI chat that scaffolds and edits components, plus branching and one-click deploy.
AI Infra2024
Disaggregated LLM serving architecture that splits prefill and decode into separate clusters and pools spare CPU, DRAM, and SSD into a distributed KVCache. Powers Kimi in production, handling 75% more requests under the same SLOs.
Official code companion to the O'Reilly book by Jay Alammar and Maarten Grootendorst: 12 chapters of runnable notebooks on tokens, embeddings, Transformers, text classification, clustering, prompt engineering, semantic search, RAG, and fine-tuning.
AI Infra2024
Provides a GenLayer DApp boilerplate implementing a football-bets intelligent contract with web and LLM integration. Bundles fast in-memory direct tests with web/LLM mocking, a GenVM linter, integration test workflows, CI, and a Next.js 15 TypeScript frontend.
Publishes a structured open textbook on large language model foundations, covering language modeling, LLM architectures, prompt engineering, PEFT, model editing, and RAG.
AI API2024
aisuite is a lightweight Python library that provides a unified API for working with multiple Generative AI providers. It supports models from OpenAI, Anthropic, Google, Hugging Face, AWS, Cohere, Mistral, Ollama, and others—abstracting away SDK differences, authentication details, and parameter variations. Modeled after OpenAI’s API style, it enables developers to build LLM-based or agentic applications across providers with minimal setup.