Serves interactive, long-lived streaming video-generation sessions by jointly scheduling session placement and GPU autoscaling to meet tight per-chunk latency. Combines migration-aware placement, load-driven autoscaling, coalesced chunk processing, GPU–CPU offloading and NCCL GPU–GPU migration; reports ~37% reductions in worst-case per-chunk latency and GPU operating cost.
A 9B reasoning LLM fine-tuned from Qwen3.5 that ships with a 1,048,576-token context, native function-calling and tool-use, and notable benchmark gains (+34 MMLU, +30 gsm8k-strict).
Provides a pre-quantized NVFP4 checkpoint of GLM-5.2 for long-context reasoning and coding; reduces model footprint so GLM-5.2 can run on multi‑GPU Blackwell nodes and is ready for inference with SGLang and vLLM.
NVFP4-quantized variant of Qwen3.6-27B that reduces parameter bits from 16 to 4, cutting disk and GPU memory requirements by ~2.5× while keeping comparable benchmark accuracy; ready for vLLM-based inference on NVIDIA hardware and supports long, multimodal contexts.
Provides an open-source Mixture-of-Experts coding LLM (397B) optimized for agentic, tool-enabled coding workflows with a 262,144-token context window, OpenAI-compatible API, serving recipes (vLLM/SGLang), and published coding-benchmark results.
230M-parameter multilingual instruction-tuned text-only LLM for on-device agentic pipelines and data extraction; 32K context, 19T-token pretraining, optimized for fast CPU/edge inference (e.g., 213 tok/s on Galaxy S25 Ultra, 42 tok/s on Raspberry Pi 5); not for heavy reasoning or complex code generation.
Local English text-to-waveform TTS producing a single fixed synthetic voice in a deployable package below 10M parameters. Offers deterministic seeds, punctuation-aware long-text chunking, CPU/CUDA and ONNX runtime options, measured evaluations and a compact FP32 footprint; English-only, one voice.
A self-improving, agentic coding LLM tailored for terminal-style coding agents and tool-calling, provided as 35B MoE GGUF weights with very large context support. Trained with reinforcement learning to jointly generate task scaffolds and solutions; designed for local inference and OpenAI-compatible tool endpoints.
Performs zero-shot classification and regression on mixed numerical and categorical tabular data by treating training rows as in-context examples and predicting in a single forward pass. Uses alternating row/column attention and row compression; limited to 10 classes and model weights are non-commercial.
Provides GGUF/llama.cpp quantized variants of Qwen3.6-27B for local multimodal inference, tuned via online RL to cut average 'thinking' tokens by ≈50% while preserving answer quality; offers Q4_K_M/Q8_0/f16 builds and a separate mmproj for vision input.
Provides quantized GGUF weights and configs for Agents‑A1 — a 35B Mixture-of-Experts agent trained for long-horizon, tool-enabled reasoning; supports 262K-context serving and runtimes like vLLM and SGLang.
Predicts per-request MoE expert footprints from prefill activations and routes decode requests to workers that maximize expert-locality, lowering decode latency by combining offline K-means partitioning with online locality-band routing and a KV-block–coindexed signature cache.