AIAny - AI Deploy

Foundry Local

2024

Microsoft

Foundry Local is an open-source tool from Microsoft that enables running generative AI models on local devices without an Azure subscription. It supports on-device processing for privacy and security, integrates models via an OpenAI-compatible API, and optimizes performance using ONNX Runtime and hardware acceleration.

microsoft ai-inference ai-serving llm ai-client+2

Agent Starter Pack

2025

Google Cloud Platform

Agent Starter Pack is a Python package providing production-ready templates for GenAI agents on Google Cloud. Focus on your agent logic—the starter pack provides everything else: infrastructure, CI/CD, observability, and security. Includes pre-built templates like ReAct, RAG, multi-agent, and more.

google ai-agent gemini RAG mlops+1

LEANN

2025

Yichuan Wang, Zhifei Li +1

LEANN is the world's smallest vector index, enabling RAG on everything from documents to chat histories with 97% storage savings, no accuracy loss, and 100% privacy on personal devices. It uses graph-based selective recomputation and high-degree preserving pruning for lightweight, scalable semantic search.

RAG github ai-library ai-tools ai-development+5

WhisperLiveKit

2025

QuentinFuxa

WhisperLiveKit is an ultra-low-latency, self-hosted speech-to-text toolkit with speaker identification. Powered by leading simultaneous speech research like Simul-Whisper and WhisperStreaming, it enables intelligent buffering and incremental processing for real-time transcription, translation across 200 languages, and speaker diarization. Ideal for meeting notes, accessibility tools, and content creation.

github ai-tools ai-inference ai-serving ASR+2

LightX2V

2025

LightX2V Contributors, ModelTC

LightX2V is an advanced lightweight video generation inference framework engineered to deliver efficient, high-performance video synthesis solutions. This unified platform integrates multiple state-of-the-art video generation techniques, supporting diverse generation tasks including text-to-video (T2V) and image-to-video (I2V). X2V represents the transformation of different input modalities (X, such as text or images) into video output (V).

github ai-video ai-tools ai-inference huggingface+2

Ray

2017

RISELab (UC Berkeley), Anyscale Inc.

Ray is an open-source distributed compute engine that lets you scale Python and AI workloads—from data processing to model training and serving—without deep distributed-systems expertise.

ai-development ai-framework ai-train ai-serving

KServe

2018

KServe Community

CNCF-incubating model inference platform (formerly KFServing) that provides Kubernetes CRDs for scalable predictive and generative workloads.

ai-development ai-inference ai-serving

Triton

2018

NVIDIA

Open-source, high-performance server for deploying and scaling AI/ML models on GPUs or CPUs, supporting multiple frameworks and cloud/edge targets.

ai-development ai-inference ai-serving nvidia

OpenVINO

2018

Intel

OpenVINO is an open-source toolkit from Intel that streamlines the optimization and deployment of AI inference models across a wide range of Intel® hardware.

ai-development ai-inference ai-serving