AI Infra2023
Puts OpenAI-, Anthropic- and Ollama-compatible endpoints in front of 60+ inference backends, so existing client code runs unchanged against local models for text, vision, audio, image and embeddings. Runs CPU-only or accelerated, data stays local.
AI Deploy2023
Provides a unified platform for AI development and deployment, including the MAX serving framework and the Mojo systems programming language for writing kernels. Offers an OpenAI-compatible inference endpoint, Mojo-written CPU/GPU kernels, and tools to run hundreds of open models across diverse hardware without vendor lock-in.
AI Infra2023
Probes LLMs for failure modes — prompt injection, jailbreaks, data leakage, toxicity, hallucination — the way nmap scans a network. Ships 20+ attack probes that run against Hugging Face, OpenAI, Bedrock, Cohere, or any REST endpoint.
AI Infra2023
Tracks, evaluates, and debugs LLM applications with traces, prompt management, datasets, playgrounds, and observability that can run in cloud or self-hosted setups.
AI Infra2023
Runs 70B-class LLM inference on a single 4GB GPU without quantization and supports Llama3.1 405B on 8GB VRAM. Uses layer-splitting and block-wise model compression (4/8-bit) to reduce disk load and can speed up inference loading by up to ~3x; integrates with Hugging Face models.
AI Image2023
Reference implementation for Stability AI's diffusion models: SDXL base/refiner/Turbo for text-to-image, plus Stable Video Diffusion, SV3D, and SV4D for image-to-video and 4D synthesis. A modular engine separates samplers, guiders, and conditioners.
AI Image2023
stable-diffusion.cpp is a pure C/C++ implementation for diffusion model inference, based on ggml, supporting models like Stable Diffusion (SD1.x, SD2.x, SDXL), Flux, Wan, Qwen Image, Z-Image, and more. It's lightweight with no external dependencies, supports backends like CPU, CUDA, Vulkan, Metal, and features like LoRA, ControlNet, LCM for efficient local image generation on platforms including Linux, Mac, Windows, and Android.
AI Train2023
Trains LLMs with RLHF at scale by splitting actor, critic, reward, and reference models across separate GPU groups via Ray, with vLLM-accelerated generation and DeepSpeed ZeRO-3. Supports PPO, GRPO, REINFORCE++, DPO, plus async and agentic multi-turn RL.
MLOps2023
Open-source AI coding assistant for VS Code and JetBrains that bundles autocomplete, chat, inline edit, and an agent mode behind one config, letting each capability use any model provider rather than a single locked-in vendor.
AI Agent2023
Gives AI agents persistent long-term memory: ingests documents in any format and continuously builds a self-hosted knowledge graph fusing vector embeddings, graph reasoning, and ontology grounding, so agents recall and reason over connected facts.
AI Infra2023
Turns any website into structured data or an API without code: record clicks once to capture lists and tables, or describe fields in plain language for AI extraction. Also crawls full sites, scrapes pages to Markdown, and runs filtered searches.
AI Infra2023
Provides a self-hosted inference engine that serves all models an agent needs—embeddings, retrieval/reranking, document-to-markdown OCR, structured extraction, content-safety scoring, and LLM generation—through an OpenAI-compatible API. Bundles a 100+ model catalog, SDKs, and production deployment tooling.