Provides unified model definitions and a single API for pretrained text, vision, audio, and multimodal models for both training and inference. Emphasizes cross-framework compatibility (PyTorch/TF/JAX), pipeline-based inference, and direct access to 1M+ Hub checkpoints.
Train and experiment with multi-billion to trillion-parameter transformer models on large GPU clusters using GPU-optimized building blocks and reference training scripts; offers advanced parallelism and mixed-precision support for research teams and ML engineers.
Provides a toolkit and codebase for building, training, and deploying speech and multimodal models — Automatic Speech Recognition, Text-to-Speech, and speech-aware LLMs — with modular neural components and pre-trained checkpoints for PyTorch. Supports streaming/low-latency inference, multi-language models, and optional compiled kernels for acceleration.
Provides vector search, LLM orchestration and language-model workflows with an embeddings database, RAG pipelines, multimodal indexing and agents. Runs locally or in containers and supports multiple models and language bindings (JS/Java/Rust/Go).
Typed Python client for the OpenAI REST API that offers synchronous and asynchronous clients, typed request/response models, streaming and Realtime support, webhook verification, and integrations for Azure and Amazon Bedrock—built for production integrations and automation.
Learns a joint image–text embedding via contrastive pretraining to enable zero-shot image classification. Uses a ViT‑B/32 image encoder and transformer text encoder; intended primarily for research into robustness and generalization, not untested deployment.
Extensible infinite-canvas React SDK for building whiteboards, diagramming tools and canvas apps — includes real-time multiplayer sync, a runtime Editor API, custom shapes/tools, and canvas primitives for LLM/AI integrations. Development free; production requires a license key.
Interactive IDE for composing AI-generated 2D and 3D scenes and producing images and videos. Provides visual scene-blocking, 3D image compositing, character posing, image-to-3D conversion, and multi-provider model integrations to make generation consistent and repeatable.
Web and desktop/mobile WebUI for generating, editing, captioning and processing images and videos with Stable Diffusion and many diffusion models. Key features include automatic model download, SDNQ on-the-fly quantization for VRAM savings, balanced CPU/GPU offload, multi-backend GPU support, and built-in captioning/tagging/upscaling workflows.
A multimodal model that accepts image and text inputs and returns text, scoring at human level on professional exams — including a bar exam in the top 10%. Its performance was forecast from models using 1/1000th the compute, showing predictable scaling.
Benchmark for evaluating general AI assistants with 466 short, real-world questions that require tool use, multimodality and reasoning; provides a public dev set and a withheld test set used for leaderboard evaluation.
Pocket-sized multimodal LLM for efficient image- and video-understanding on mobile and edge devices, featuring mixed 4x/16x visual-token compression (MiniCPM‑V 4.6), compact 1.3B variants, and ready guides for iOS/Android/HarmonyOS deployment.