Combines an Unsloth UD XL quantization of the Qwen3.6-27B base with compact, Q8_0-encoded Multi-Token Prediction (MTP) heads grafted on top. The core idea is to keep the heavy base model in an efficient dynamic quantized GGUF while preserving near-lossless MTP drafts in Q8_0 to enable speculative / multi-token decoding without fully requantizing the MTP heads.
What Sets It Apart
- Quantization + grafted MTP: uses Unsloth Dynamic 2.0 UD XL for the main weights and merges separate Q8_0 MTP layers, which means you get the inference-memory savings of UD XL while retaining effective MTP drafts. So what: lower memory/throughput cost for enabling speculative decoding compared with keeping all MTP layers at full precision.
- Practical compatibility note: designed to run with llama.cpp builds that include MTP/speculative-decoding support (the model card references PR #22673). So what: you can run speculative decoding locally with llama-server if you follow the custom-build requirement; otherwise the grafted MTP heads will not be used.
- Targeted pipeline and tradeoffs: labeled for image-text-to-text and built from Qwen3.6-27B artifacts. So what: it’s suitable for multimodal inference scenarios that benefit from multi-token drafts, but it is a specialized build (GGUF + grafted heads) rather than an official upstream release.
Who It's For and Trade-offs
Great fit if you need a quantized Qwen3.6 variant that can leverage multi-token/speculative decoding for higher throughput in multimodal (image+text) tasks and you can run or build a llama.cpp runtime with MTP support. Look elsewhere if you need a drop-in Hugging Face Transformers model or managed serving that uses standard inference stacks without custom speculative-decoding patches — those environments may ignore the grafted MTP heads. Also note this repository stores the merged GGUF artifacts and conversion script; it intentionally trades some operational simplicity for a smaller MTP memory footprint (Q8_0) to avoid requantizing the full model.
