Making full 27B-class reasoning practical on laptops and single GPUs is the project's central insight: by encoding the entire language-model weights as ternary values with group-wise FP16 scales, the release cuts the deployed footprint dramatically while keeping most reasoning, math and coding performance intact.
Key Capabilities
- Aggressive low-bit representation: end-to-end ternary weights (Q2_0_g128) with an information-theoretic cost of ~1.71 bits/weight and a deployed GGUF pack around ~7.2 GB. This yields an ideal ~9.4× reduction vs FP16.
- High retained capability: measured at ~80.49 average across 15 thinking-mode benchmarks (~94.6% of FP16 average), with math (~93.4) and coding (~85.96) close to full precision.
- Long on-device context: supports a 262K-token window via a hybrid-attention backbone and 4-bit KV-cache quantization; peak memory for 100K context fits mainstream laptops (~14.7 GB without KV compression, ~10.1 GB with 4-bit KV compression).
- Practical serving and kernels: ships in GGUF Q2_0_g128 for llama.cpp and provides custom low-bit kernels (PrismML forks) for CUDA and Metal; includes an optional DSpark speculative drafter to speed decoding on CUDA.
- Complementary builds: 1-bit companion exists for phone-class deployments (~3.9 GB), and an MLX variant targets native Apple Silicon.
Who it's for and tradeoffs
Great fit if you need a 27B-capability agent on laptop or a single commodity GPU, want long-context document/agent workflows (up to 262K tokens), or require on-device/privacy-preserving inference without cloud costs. Look elsewhere if you require absolute FP16 parity (there is a small, predictable gap concentrated on the toughest reasoning/instruction-following subsets), strict per-app phone memory budgets (ternary pack > ~6 GB per-app iOS), or if your deployment stack cannot run custom low-bit kernels (native ternary kernels are still evolving).
Where it sits vs alternatives
Compared with conventional Q4/Q2 4-bit/2-bit builds, the ternary operating point targets quality-first low-bit deployment: smaller footprint than typical 2–4-bit packs while avoiding the selective collapse of reasoning seen in many sub-4-bit methods. The tradeoff is engineering reliance on custom low-bit kernels and currently shipped 2-bit slots (deployed bytes modestly above the theoretical minimum).
If you evaluate models for on-device agents, this release is a decision point: it converts many previously impractical 27B use cases into feasible local deployments, while preserving a clear and measurable, but nonzero, accuracy gap vs FP16.
