AI Infra2020
Covers the full AI quant pipeline — point-in-time data, model training, backtesting, portfolio optimization, and order execution. Supports supervised learning, market dynamics, and RL on 20+ models, plus an LLM-based RD-Agent for factor mining.
AI Infra2020
Unified framework for few-shot evaluation of generative language models across 60+ academic benchmarks. Supports multiple model backends (Hugging Face, vLLM, APIs, local servers), configurable prompts/YAML configs, and reproducible exports for leaderboards and research comparisons.
AI Infra2020
Sits between PyTorch and micrograd: eager tensors with autograd plus a small, fully hackable compiler that fuses operations into kernels. Adding a new accelerator backend takes about 25 low-level ops, so it runs on CUDA, Metal, AMD, and WebGPU.
AI Deploy2021
Deploys trained SavedModels behind gRPC and REST endpoints, with hot-swappable versioning so new weights load without downtime. Built around servables, loaders, sources, and a manager, plus request batching to cut accelerator cost.
MLOps2021
Unified metadata platform for data discovery, observability, and governance — central metadata repository, column-level lineage, and a pluggable ingestion framework with 84+ connectors. Suited for teams that need searchable data catalogs, automated lineage, and collaborative data governance.
AI Infra2021
Runs, manages, and scales AI workloads across 20+ clouds, Kubernetes, Slurm, and on-prem from one YAML or Python spec. Auto-provisions GPUs/TPUs, fails over across regions and providers when capacity is short, and routes jobs to the cheapest option.
AI Deploy2021
Serves predictive and generative ML models on Kubernetes via a single InferenceService CRD, with scale-to-zero, canary rollouts, and an OpenAI-compatible LLM path on vLLM. One autoscaling abstraction over PyTorch, XGBoost, ONNX, and HuggingFace.
AI Infra2021
Drop-in transformer building blocks with custom CUDA kernels: memory-efficient exact attention (up to ~10x faster), block-sparse attention, fused softmax/layernorm/SwiGLU ops. Cuts VRAM and speeds up diffusion and LLM training on Nvidia GPUs.
AI Train2021
HPC-AI Technology Inc. (Colossal-AI team), Shenggui Li +1HPC-AI Technology Inc., National University of Singapore
Scales a single-GPU training script to thousands of GPUs through a unified interface, combining data, pipeline, tensor, and sequence parallelism. Its Gemini memory manager offloads tensors across GPU, CPU, and NVMe so models far larger than VRAM still fit.
AI Infra2022
A continuously-updated, categorized self-hosting guide that catalogs tools, deployment notes and resources for containers, networking, home automation and running LLMs/chatbots locally; provided as a long, structured README with links and quick setup tips.
AI Infra2022
Compiles plain Python functions into GPU or CPU kernels at runtime via a JIT decorator, with differentiable output that plugs into PyTorch, JAX, and Paddle. Ships physics, robotics, geometry, and FEM primitives — particles, meshes, ray-casting, FFT.