Large 27B models are frequently impractical for local use because blunt low-bit dumps damage tensors that the architecture depends on. This Ridge release targets that problem: it deliberately preserves higher precision for Gated-DeltaNet state and critical norms while aggressively lowering bits elsewhere, enabling an interactive 27B-weighted runtime that still fits modest GPUs.
What Sets It Apart
- GDN-aware quantization: sensitive Gated-DeltaNet state (ssm_alpha/ssm_beta) is kept at higher precision (Q8_0) while mixers and many matrices use lower-bit formats. This avoids the large quality regressions typical of flat 2-bit dumps. The measured bitrate is 3.69 bits-per-weight (11.73 GiB) with a wiki-style PPL ~9.3% worse than a BF16 convert.
- MTP and vision preserved: the native MTP draft head remains embedded (no imatrix), and image input is supported via a separate BF16 mmproj (~0.87 GiB) so vision capability is optional and kept separate from the text GGUF.
- Practical hardware sizing: weights are small enough (~11.7 GiB) to be comfortably interactive on 16–24 GB GPUs at modest context lengths; measured throughput on a RTX PRO 6000 Blackwell (96 GB) was ~54 tok/s generation and ~130 tok/s prompt in llama.cpp CUDA tests.
Key Capabilities
- Enables local inference with a full Qwen3.8-27B topology while reducing weight size to ~11.73 GiB (3.69 bpw).
- Retains long-context support (native 262,144 tokens, extensible to 1,000,000 with YaRN); caution that KV cache, not weights, drives VRAM usage at long contexts.
- Works with common GGUF runtimes (llama.cpp, Ollama, LM Studio, jan, KoboldCpp) and supports draft-MTP speculation in runtimes that implement it.
Who It's For and Tradeoffs
- Great fit if you need an interactive local 27B-class model on a single 16–24 GB GPU and want to avoid the accuracy penalties of naive ultra-low-bit quantization. Good for multimodal workflows if you add the mmproj file.
- Look elsewhere if you require lossless parity with BF16 quality (this release is not lossless; measured ~+9% wiki-style PPL vs BF16) or if your runtime lacks MTP/draft support (you'll lose the MTP speedup). Also plan for substantial KV-cache memory when using the model's very long context windows.
Overall, the Ridge GGUF is a pragmatic trade: it recovers much of the accuracy lost by flat 2-bit conversions by protecting architecture-sensitive tensors, delivering a usable 27B on constrained GPU memory while making clear the quality vs. size compromises.
