AI Infra2024
Serves large language and multimodal models with low latency and high throughput using RadixAttention, continuous batching, structured outputs, parallelism, quantization, and broad accelerator support.
AI Agent2024
Curated developer resources that demonstrate building RAG systems, multi-agent workflows, and memory-augmented AI using Oracle AI Database and OCI — includes end-to-end reference apps, notebooks, guides, and workshops for hands-on prototyping.
AI Infra2024
Runs AI-generated code in isolated, elastic sandboxes with SDK, API, and CLI access for agent workflows that need stateful execution and environment control.
AI Infra2024
Builds a knowledge graph from a text corpus by extracting entities and relations, clusters it into communities with the Leiden algorithm, and summarizes them — so queries can synthesize across scattered documents instead of retrieving isolated chunks.
AI Infra2024
Open-weights 314B-parameter Mixture-of-Experts language model (8 experts, 2 active per token, 8,192-token context) released under Apache 2.0. Ships a raw JAX checkpoint plus reference inference code; needs heavy multi-GPU memory to load.
AI Infra2024
A minimal GPU written in under 15 SystemVerilog files to teach how GPUs execute parallel kernels from the ground up. Includes an 11-instruction ISA, multiple cores with ALUs and load-store units, a fetch-decode-execute pipeline, and matrix kernels.
AI Infra2024
Chains pre-trained AI weather and climate models like GraphCast, Pangu, and FourCastNet into composable inference pipelines. Swap prognostic or diagnostic components, plug in reanalysis sources, and add ensemble perturbations or in-loop metrics.
AI Train2024
Accelerates video generation with a unified framework for inference, finetuning, LoRA, distillation, sparse attention, and distributed execution for research and demos.
AI Infra2024
Stores and reuses LLM key-value caches across GPU, CPU, disk, and remote backends so vLLM and SGLang skip recomputing repeated context. Non-prefix reuse (CacheBlend) and PD disaggregation cut time-to-first-token for long-context and RAG serving.
AI Infra2024
Cloud-native control plane that scales vLLM on Kubernetes, adding the routing, autoscaling, and fault tolerance single-instance serving lacks. Brings high-density LoRA management, an LLM gateway, distributed KV cache reuse, and SLO-aware GPU serving.
AI Infra2024
Connects multiple Macs and Linux machines into one cluster to run models too large for any single machine. Auto-discovers peers, shards a model across them via tensor parallelism, and exposes OpenAI-, Claude-, and Ollama-compatible APIs.
AI Infra2024
Disaggregated LLM serving architecture that splits prefill and decode into separate clusters and pools spare CPU, DRAM, and SSD into a distributed KVCache. Powers Kimi in production, handling 75% more requests under the same SLOs.