AI Train2024
Streamlines the full lifecycle of foundation models — data prep, fine-tuning (SFT/LoRA/QLoRA/GRPO), evaluation, and deployment — with ready-to-run recipes, multi-engine inference support, and cloud/CLI workflows for both laptop experiments and large-scale runs.
AI Agent2024
Splits autonomous R&D into two cooperating agents: one proposes hypotheses, the other writes and tests code — iterating on quant-finance factors, Kaggle pipelines, and model research. Hits a ~30% medal rate on MLE-Bench, nearly double AIDE's.
MLOps2024
Runs reproducible evaluations of large language models through a Python API with built-in solvers, scorers, and model-graded grading. Ships 200+ ready-to-run evals spanning capability and safety testing, and connects to most major model providers.
AI API2024
Build AI workflows once and run them across model providers — GoogleAI, OpenAI, Claude, Ollama — through one SDK. Composable primitives for RAG, tool use, and agents, plus a local dev UI for tracing and debugging, with SDKs in JS/TS, Go, and Python.
AI Infra2024
Stores and reuses LLM key-value caches across GPU, CPU, disk, and remote backends so vLLM and SGLang skip recomputing repeated context. Non-prefix reuse (CacheBlend) and PD disaggregation cut time-to-first-token for long-context and RAG serving.
AI Infra2024
Cloud-native control plane that scales vLLM on Kubernetes, adding the routing, autoscaling, and fault tolerance single-instance serving lacks. Brings high-density LoRA management, an LLM gateway, distributed KV cache reuse, and SLO-aware GPU serving.
AI Infra2024
Connects multiple Macs and Linux machines into one cluster to run models too large for any single machine. Auto-discovers peers, shards a model across them via tensor parallelism, and exposes OpenAI-, Claude-, and Ollama-compatible APIs.
AI Infra2024
Provides a GenLayer DApp boilerplate implementing a football-bets intelligent contract with web and LLM integration. Bundles fast in-memory direct tests with web/LLM mocking, a GenVM linter, integration test workflows, CI, and a Next.js 15 TypeScript frontend.
AI Infra2024
kvcache-ai (maintainers / MADSys group, Tsinghua University & contributors)MADSys Lab, Tsinghua University, Approaching.AI +1
Runs huge mixture-of-experts LLMs like DeepSeek-R1/V3 on a single 24GB GPU plus CPU DRAM by keeping attention on the GPU and offloading expert weights to CPU. Reports 3-28x speedups via Intel AMX/AVX512 kernels and fits 139K context in 24GB VRAM.
AI Agent2024
Runs autonomous AI-agent workforces where each agent, skill, and company process lives as version-controlled code you own. Agents act in isolated sandboxes and submit deliverables for human review, with 3,000+ connectors plus MCP support.
AI Train2024
Parses, generates, and filters training data from noisy sources like PDFs and weak QA, then feeds it into LLM pre-training, SFT, RL, or RAG cleaning. Ships 100+ operators and ready-made pipelines for text, reasoning, Text2SQL, and agentic data.
A GitHub repository of learning notes and code dedicated to ML + SYS (machine learning systems). It collects tutorials, code walkthroughs and engineering notes on RLHF, distributed training (FSDP, Megatron), inference and scheduling (SGLang, vllm), quantization, CUDA/GPU optimization, system design, and practical engineering.