Tag
Explore by tags
Runs LLM-generated Python in a Rust sandbox that starts in tens of microseconds (~60µs), with no container overhead. Filesystem, network, and environment access are blocked, and state serializes for pause/resume with per-run resource limits.
Teaches generative AI app development through 21 lessons covering LLM basics, prompting, chat, search, image generation, agents, RAG, fine-tuning, small models, and responsible AI.
Turns local documents into a private, self-hosted ChatGPT-style assistant with no-code agents for web browsing and workflow automation. Runs across LLM providers — OpenAI, Anthropic, Ollama — and routes tools smartly to cut token use.
Runs 70B-class LLM inference on a single 4GB GPU without quantization and supports Llama3.1 405B on 8GB VRAM. Uses layer-splitting and block-wise model compression (4/8-bit) to reduce disk load and can speed up inference loading by up to ~3x; integrates with Hugging Face models.
Run any open-source LLM, embedding, speech, image, or multimodal model behind one OpenAI-compatible API — swap GPT for an open model in a single line. Routes across vLLM, llama.cpp, GGML, and TensorRT, scaling from a laptop to a multi-node GPU cluster.
Unified TypeScript toolkit for building AI apps and agents: swap between 100+ models from OpenAI, Anthropic, Google and others by changing one line. Ships streaming, tool calling, and framework-agnostic UI hooks for React, Next.js, Vue, and Svelte.
Compresses, deploys, and serves LLMs via two engines: TurboMind for raw speed, a PyTorch engine for flexibility. Claims ~1.8x vLLM throughput through persistent batching, blocked KV cache, and split-and-fuse; ships 4-bit AWQ and KV-cache quantization.
Adds agent-native UI patterns to apps through chat, generative UI, shared state, human-in-the-loop flows, and AG-UI-based frontend integrations.
Orchestrates LLM-based roles (product managers, architects, engineers) to turn a one-line requirement into user stories, APIs and a starter code repo. SOP-driven multi-agent workflows with CLI and library APIs for prototype generation and agentic development.
Upload your own documents, PDFs, slides, or web pages and ask questions answered only from that material, with inline citations pointing to the exact passage used. Audio Overview turns your sources into a downloadable two-host podcast discussion.
Lets LLMs run code and control a user’s computer via natural language (Python, JavaScript, Shell, etc.) with interactive approval. Supports local or hosted models, terminal and Colab/Codespaces integrations, streaming output, and configurable safety/auto-run options.
Calls 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure — through one OpenAI-compatible API, as a Python SDK or self-hosted proxy. The proxy adds virtual keys, spend tracking, rate limits, and load balancing across models and providers.