Tag
Explore by tags
Builds and deploys machine learning models across research, production, web, mobile, and edge environments. Its ecosystem spans Keras, TFX, LiteRT, TensorFlow.js, datasets, model hubs, and visualization tools.
Serves machine learning and deep learning models for cloud, data center, edge and embedded environments. Supports multiple frameworks and backends, dynamic and sequence batching, HTTP/gRPC APIs, Docker deployment and NVIDIA-optimized runtimes.
Runs ONNX models faster on CPU, GPU, and NPU by routing graph subgraphs to backend execution providers (CUDA, TensorRT, OpenVINO, DirectML, CoreML). One engine serves the same model across cloud, browser, mobile, and edge, for both inference and training.
Train and experiment with multi-billion to trillion-parameter transformer models on large GPU clusters using GPU-optimized building blocks and reference training scripts; offers advanced parallelism and mixed-precision support for research teams and ML engineers.
Serves predictive and generative ML models on Kubernetes via a single InferenceService CRD, with scale-to-zero, canary rollouts, and an OpenAI-compatible LLM path on vLLM. One autoscaling abstraction over PyTorch, XGBoost, ONNX, and HuggingFace.
Rust-and-Python toolkit that serves open-source LLMs (Llama, Falcon, Mixtral, StarCoder) over HTTP/gRPC with tensor parallelism, continuous batching, Flash/Paged Attention and quantization. Now in maintenance mode, pointing users toward vLLM and SGLang.
Run any open-source LLM, embedding, speech, image, or multimodal model behind one OpenAI-compatible API — swap GPT for an open model in a single line. Routes across vLLM, llama.cpp, GGML, and TensorRT, scaling from a laptop to a multi-node GPU cluster.
GPU kernel library for LLM inference attention, sampling, and KV-cache, built on block-sparse formats with JIT-compiled customizable templates. Reports 29-69% inter-token-latency cuts vs compiler backends; powers SGLang, vLLM, and MLC-Engine.
Applies quantization, pruning, distillation, speculative decoding and sparsity to compress and accelerate deep learning models, producing exportable checkpoints ready for deployment on TensorRT‑LLM, TensorRT, vLLM and similar inference runtimes.
A research codebase and model family for vision–language models that experiments with data‑centric post‑training strategies and long‑context multimodal reasoning. Includes model reports, released research weights (non‑commercial), grounding tools (LocateAnything) and integrations for inference/optimization.
A 671B-parameter Mixture-of-Experts language model (37B activated) trained on 14.8T tokens with 128K context, FP8-first training, a Multi-Token Prediction module, and Hugging Face weights—focused on efficient MoE training and long-context use cases.