Tag

Ollama

Run and manage open and community LLMs locally via a compact CLI and REST API—supports model import, Docker deployment, and official Python/JS SDKs for local inference, RAG, and dev workflows.

Elasticsearch

Distributed search and analytics engine and vector database built on Lucene that enables near-real-time full-text and vector search, indexing, and analytics over large datasets. Provides vector embeddings support, REST APIs, RAG-friendly features, and deployment options including Elastic Cloud and Docker.

Programmatically author, schedule, and monitor data workflows as Python-defined DAGs; the scheduler handles dependencies, retries, and backfills. Pluggable executors (Local, Celery, Kubernetes) and a broad provider ecosystem for AWS, GCP, and databases.

TensorFlow

Builds and deploys machine learning models across research, production, web, mobile, and edge environments. Its ecosystem spans Keras, TFX, LiteRT, TensorFlow.js, datasets, model hubs, and visualization tools.

TensorRT

Converts trained PyTorch, TensorFlow, and ONNX models into GPU-tuned inference engines via layer fusion, kernel auto-tuning, and reduced precision. Cuts latency, raises throughput on NVIDIA GPUs from Turing (INT8), with FP8 on Ada+ and FP4 on Blackwell+.

ONNX

Defines a portable model format and operator set for moving trained machine learning models across frameworks, runtimes, and hardware targets without locking the model to one toolchain.

Ray (by Anyscale)

Scales any Python or ML workload across CPUs and GPUs with a few decorators, instead of rewriting code for Spark or MPI. Bundles libraries for distributed training, hyperparameter tuning, RL, batch inference, and online model serving on one cluster.

Triton Inference Server

Serves machine learning and deep learning models for cloud, data center, edge and embedded environments. Supports multiple frameworks and backends, dynamic and sequence batching, HTTP/gRPC APIs, Docker deployment and NVIDIA-optimized runtimes.

OpenVINO

Converts, quantizes, and runs deep learning models from PyTorch, TensorFlow, ONNX, and PaddlePaddle across Intel CPUs, GPUs, and NPUs without the training framework. Adds a GenAI pipeline for LLMs plus Hugging Face, vLLM, and LangChain integrations.

ONNX Runtime | Home

Runs ONNX models faster on CPU, GPU, and NPU by routing graph subgraphs to backend execution providers (CUDA, TensorRT, OpenVINO, DirectML, CoreML). One engine serves the same model across cloud, browser, mobile, and edge, for both inference and training.

Bento: Run Inference at Scale

Turns Python ML code into production inference APIs that scale on Kubernetes or any cloud. Bundles models, dependencies, and serving logic into versioned "Bentos" with autoscaling, scale-to-zero, and multi-GPU serving for LLMs and custom models.

Milvus

Runs approximate nearest-neighbor search over billions of vector embeddings, separating compute from storage so reads and writes scale independently. Offers HNSW, IVF, DiskANN, and GPU CAGRA indexes plus hybrid dense+sparse and BM25 retrieval.