AI Model2026
Provides a GGUF-quantized build of NVIDIA's Nemotron 3 Nano Omni 30B (Reasoning) for local inference — enables multimodal (video/audio/image/text) reasoning, transcription, and document understanding on compatible runtimes such as llama.cpp, Ollama, vLLM, and TensorRT-LLM.
AI Deploy2026
Provides reusable Docker compose files, scripts and benchmarked configs to serve modern LLMs (Qwen, Gemma, etc.) on 1–2 NVIDIA RTX 3090/4090/5090 GPUs. Multi-engine (vLLM, llama.cpp, ik_llama), measured TPS/context tradeoffs, and validated single/dual‑GPU recipes.
AI Model2026
An instruct-focused LLM (104B total, 7.4B active) optimized for fast, token-efficient inference in agent workflows. Uses hybrid linear attention plus a sparse MoE to raise throughput and cut token use; suited for high-frequency production agents, with some trade-offs in very deep reasoning.
AI Model2026
A trillion-parameter LLM optimized for long-context, low-latency text generation and agentic coding workflows. Combines MLA+Linear Attention and a post-training 'fast thinking' token-suppression strategy to reduce token overhead and improve multi-step execution reliability for production agents.
AI Model2026
DavidAU/Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking-NEO-CODE-Di-IMatrix-MAX-GGUF
A 40B GGUF-quantized Qwen3.6 variant fine-tuned with Claude 4.6 Opus and Deckard/Heretic datasets for multimodal image-text-to-text tasks. Offers 256K context, custom NEO-CODE Di-IMatrix quants for long conversations and coding, optimized for local inference and creative/coding use cases; safety alignment removed.
AI Deploy2026
Runs local LLMs on Apple Silicon using native MTP speculative decoding to accelerate token generation while preserving the model's output distribution. Leverages the model's own MTP heads with batched verification and exact rejection sampling; ships with a Mac app, CLI, local OpenAI/Anthropic-compatible server, auto-tune, and Forge for building/verifying MTP adapters.
AI Infra2026
Native local inference engine for DeepSeek V4 Flash (also supports GLM 5.2 and PRO on high‑memory machines). Focused features include model-specific loading, SSD expert streaming, asymmetric routed-expert 2-bit quant support, multi-GPU/tensor/pipeline parallelism, and an OpenAI-compatible server plus a native coding agent.
AI Model2026
Merges Unsloth UD XL quantized GGUF of Qwen3.6-27B with compact Q8_0 MTP heads to enable multi-token (speculative) decoding on llama.cpp builds that support MTP; aimed at image-text-to-text usage with reduced MTP overhead.
AI Infra2026
High-throughput LLM inference engine for agentic workloads, combining a local‑SPMD static compiler for parallelism, a C++ scheduler with a Python execution plane and type‑safe KV‑cache reuse, pluggable high-performance kernels (including an MLA implementation), and a low‑overhead AsyncLLM entrypoint for production GPU inference.
AI Audio2026
Converts text into natural-sounding speech locally using compact ONNX TTS assets. Optimized for CPU/edge inference (~99M params) with support for 31 languages, expression tags (e.g., <laugh>), and improved stability versus Supertonic 2 — suitable for on-device multilingual TTS.
AI Model2026
7B multilingual translation model optimized for instruction-following and low-latency deployment across 33 languages; provides quantized/FP8/GGUF builds and integrations (vLLM, llama.cpp) for server and on-device inference.
AI Model2026
Provides a quantized GGUF build of Qwen3.6‑27B with MTP (multi‑token prediction) support for faster local inference. Packaged for GGUF-compatible runners (llama.cpp) and Hugging Face/transformers workflows, with deployment notes for CPU/GPU and vLLM/SGLang integration.