Discover the Best AI Resources

promptfoo

Declarative CLI and library to evaluate and red-team LLM apps: run test cases against prompts and models, compare providers side-by-side, and scan for jailbreaks, prompt injection, and data leaks — with CI/CD and pull-request code scanning built in.

Modular Platform

Provides a unified platform for AI development and deployment, including the MAX serving framework and the Mojo systems programming language for writing kernels. Offers an OpenAI-compatible inference endpoint, Mojo-written CPU/GPU kernels, and tools to run hundreds of open models across diverse hardware without vendor lock-in.

MLC LLM

Compiles one LLM into device-native binaries running on CUDA, ROCm, Metal, Vulkan, WebGPU, and CPU — same model from server to browser to phone. On Apache TVM, it ships MLCEngine with an OpenAI-compatible API across Python, JS, REST, iOS, and Android.

PrivateGPT

Wraps a local, OpenAI-compatible inference server in one messages API so you can build private AI apps with no data leaving your network: document ingestion, retrieval with inline citations, and built-in tools (web search, code execution, MCP).

Arena Leaderboard (formerly LMArena)

Blind side-by-side voting site where users send one prompt to two anonymous chat models, pick the winner, and millions of votes become Elo rankings across text, coding, vision, image, and video. Crowd preference, not static benchmarks, decides the order.

LitGPT

Provides from‑scratch implementations and end‑to‑end workflows to pretrain, finetune, evaluate, and deploy 20+ LLMs with reproducible YAML recipes, multi‑GPU/TPU support, quantization, and LoRA/QLoRA options.

Generative AI on Google Cloud

Notebooks and sample apps demonstrating generative-AI workflows on Google Cloud's Vertex AI and Gemini — covering RAG grounding, multimodal demos, function calling, and agent-building examples, with deployment-ready templates for evaluation and production.

Ragas

Evaluates and tests LLM apps — RAG pipelines, agents, and workflows — using objective metrics that mix LLM-as-judge scoring with deterministic measures. Auto-generates synthetic test datasets and integrates with LangChain and tracing tools.

Open Generative AI

Provides an uncensored, self‑hostable studio for generating AI images, videos, and lip‑synced talking videos in browser or desktop. Integrates 200+ models via Muapi.ai, supports local inference (stable-diffusion.cpp), multi-image inputs and workflow automation — no content filters.

garak

Probes LLMs for failure modes — prompt injection, jailbreaks, data leakage, toxicity, hallucination — the way nmap scans a network. Ships 20+ attack probes that run against Hugging Face, OpenAI, Bedrock, Cohere, or any REST endpoint.

Opik

Provides end-to-end observability, evaluation, and optimization for LLM-based applications by tracing model calls, running automated evaluations, and surfacing production metrics. Ships SDKs, broad framework integrations, LLM-as-a-judge metrics, and dashboards to support development, CI, and production monitoring.

Swarms

Enterprise-grade multi-agent orchestration framework that builds, runs, and scales autonomous agent swarms for production. Offers modular swarm architectures, protocol support (MCP, AOP), a marketplace, multi-model provider integrations and observability.