AI Infra2024
Runs AI-generated code in isolated, elastic sandboxes with SDK, API, and CLI access for agent workflows that need stateful execution and environment control.
AI Infra2024
Builds a knowledge graph from a text corpus by extracting entities and relations, clusters it into communities with the Leiden algorithm, and summarizes them — so queries can synthesize across scattered documents instead of retrieving isolated chunks.
AI Infra2024
Provides a lightweight build platform for HIP and ROCm that supports building ROCm, PyTorch, and JAX from source, multi-architecture nightly releases, and integrated CI/CD and developer tooling for Linux and Windows.
AI Infra2024
Open-weights 314B-parameter Mixture-of-Experts language model (8 experts, 2 active per token, 8,192-token context) released under Apache 2.0. Ships a raw JAX checkpoint plus reference inference code; needs heavy multi-GPU memory to load.
AI Infra2024
A minimal GPU written in under 15 SystemVerilog files to teach how GPUs execute parallel kernels from the ground up. Includes an 11-instruction ISA, multiple cores with ALUs and load-store units, a fetch-decode-execute pipeline, and matrix kernels.
AI Infra2024
Chains pre-trained AI weather and climate models like GraphCast, Pangu, and FourCastNet into composable inference pipelines. Swap prognostic or diagnostic components, plug in reanalysis sources, and add ensemble perturbations or in-loop metrics.
AI Train2024
Accelerates video generation with a unified framework for inference, finetuning, LoRA, distillation, sparse attention, and distributed execution for research and demos.
AI Infra2024
Stores and reuses LLM key-value caches across GPU, CPU, disk, and remote backends so vLLM and SGLang skip recomputing repeated context. Non-prefix reuse (CacheBlend) and PD disaggregation cut time-to-first-token for long-context and RAG serving.
AI Infra2024
Cloud-native control plane that scales vLLM on Kubernetes, adding the routing, autoscaling, and fault tolerance single-instance serving lacks. Brings high-density LoRA management, an LLM gateway, distributed KV cache reuse, and SLO-aware GPU serving.
AI Infra2024
Connects multiple Macs and Linux machines into one cluster to run models too large for any single machine. Auto-discovers peers, shards a model across them via tensor parallelism, and exposes OpenAI-, Claude-, and Ollama-compatible APIs.
AI Infra2024
Disaggregated LLM serving architecture that splits prefill and decode into separate clusters and pools spare CPU, DRAM, and SSD into a distributed KVCache. Powers Kimi in production, handling 75% more requests under the same SLOs.
AI Infra2024
Provides a GenLayer DApp boilerplate implementing a football-bets intelligent contract with web and LLM integration. Bundles fast in-memory direct tests with web/LLM mocking, a GenVM linter, integration test workflows, CI, and a Next.js 15 TypeScript frontend.