AI API2024
Build AI workflows once and run them across model providers — GoogleAI, OpenAI, Claude, Ollama — through one SDK. Composable primitives for RAG, tool use, and agents, plus a local dev UI for tracing and debugging, with SDKs in JS/TS, Go, and Python.
AI Infra2024
Stores and reuses LLM key-value caches across GPU, CPU, disk, and remote backends so vLLM and SGLang skip recomputing repeated context. Non-prefix reuse (CacheBlend) and PD disaggregation cut time-to-first-token for long-context and RAG serving.
AI Infra2024
Cloud-native control plane that scales vLLM on Kubernetes, adding the routing, autoscaling, and fault tolerance single-instance serving lacks. Brings high-density LoRA management, an LLM gateway, distributed KV cache reuse, and SLO-aware GPU serving.
AI Infra2024
Connects multiple Macs and Linux machines into one cluster to run models too large for any single machine. Auto-discovers peers, shards a model across them via tensor parallelism, and exposes OpenAI-, Claude-, and Ollama-compatible APIs.
AI Infra2024
kvcache-ai (maintainers / MADSys group, Tsinghua University & contributors)MADSys Lab, Tsinghua University, Approaching.AI +1
Runs huge mixture-of-experts LLMs like DeepSeek-R1/V3 on a single 24GB GPU plus CPU DRAM by keeping attention on the GPU and offloading expert weights to CPU. Reports 3-28x speedups via Intel AMX/AVX512 kernels and fits 139K context in 24GB VRAM.
AI Agent2024
Runs autonomous AI-agent workforces where each agent, skill, and company process lives as version-controlled code you own. Agents act in isolated sandboxes and submit deliverables for human review, with 3,000+ connectors plus MCP support.
AI Train2024
Parses, generates, and filters training data from noisy sources like PDFs and weak QA, then feeds it into LLM pre-training, SFT, RL, or RAG cleaning. Ships 100+ operators and ready-made pipelines for text, reasoning, Text2SQL, and agentic data.
A GitHub repository of learning notes and code dedicated to ML + SYS (machine learning systems). It collects tutorials, code walkthroughs and engineering notes on RLHF, distributed training (FSDP, Megatron), inference and scheduling (SGLang, vllm), quantization, CUDA/GPU optimization, system design, and practical engineering.
AI Infra2025
Spins up sandboxed VMs and containers (macOS, Linux, Windows, Android) that an AI agent can fully control through one unified SDK, cloud or local, plus a benchmark suite and background drivers that automate native apps without grabbing the cursor.
AI Train2025
inclusionAI (AReaL Team), Institute for Interdisciplinary Information Sciences, Tsinghua University +1Ant Group, Institute for Interdisciplinary Information Sciences, Tsinghua University
Trains LLM reasoning and agentic models with fully asynchronous reinforcement learning, decoupling rollout generation from policy updates for a 2.77x speedup over synchronous RL. Covers GRPO, PPO and DAPO across Megatron, FSDP, vLLM and SGLang backends.
AI Agent2025
Scaffolds production-ready GenAI agents on Google Cloud from one CLI command, wrapping your agent logic in Terraform, CI/CD, observability, and evaluation. Ships ADK, LangGraph, and multimodal RAG templates for Cloud Run or Vertex AI Agent Engine.
AI Infra2025
Keeps codebases, PDFs, Slack, and docs continuously indexed for RAG and knowledge graphs by recomputing only what changed, not the whole dataset. You declare target state in Python; a Rust engine maintains it with per-row lineage back to the source.