Why this matters
This release makes a deployable, locally runnable variant of Qwen3.8-27B that keeps the model's native MTP speculative-draft head and applies a targeted weight edit to reduce safety-driven refusals. That combination preserves the speculative-decoding speedups while exposing a measured reduction in refusal behaviour, so operators can evaluate trade-offs between safety constraints and capability in a reproducible way.
Key Capabilities
- MTP preserved: the multi-token-prediction (MTP/NextN) draft head is grafted back into each shipped GGUF, enabling llama.cpp speculative decoding without a separate sidecar.
- Measured behaviour: refusal rate on a 100-item harmful prompt set dropped from 98/100 (base) to 12/100 for the published point; KL divergence vs base (first-token) is 0.1191. Benchmarks (0-shot) show a mean delta of −0.5 across MMLU, ARC-Challenge, HellaSwag and Winogrande under the test harness used.
- Multiple quantizations: files range from IQ2_M (~10.6 GB) up to Q8_0 (~29.0 GB), with published per-quant perplexities on wikitext-2 and an imatrix calibration artifact to reproduce quants.
- Speculative decoding supported and benchmarked: fused and noMTP/draft split workflows are provided, with throughput measurements and guidance for llama.cpp flags.
Method and verification
- Refusal edits were produced by Heretic, which co-optimizes refusal count and KL divergence; edits were applied at bf16 as a LoRA that was merged into the base before conversion.
- The MTP tensors are copied verbatim from the base checkpoint after the merge; every GGUF was inspected post-quantization to confirm the MTP block survives (65/65 blocks reported for fused files).
- Calibration imatrix was computed from the f16 GGUF (published imatrix included) and used to build the mixed-precision quants; the card documents procedures and the exact llama.cpp commits used.
Who it's for & trade-offs
Great fit if you need a local, reproducible deployment of Qwen3.8 that: wants MTP speculative decoding, needs multiple quant size options for tight VRAM targets, and is prepared to evaluate and accept reduced refusal behaviour under a specified safety trade-off. Look elsewhere if you require fully enforced refusal/safety guarantees, extensive generative-code/math evaluations (not provided here), or if you cannot run a recent llama.cpp build that supports MTP. Lower-bit quants (notably IQ2_M) show larger degradation and greater instability around prior refusal boundaries; evaluate on Q6_K or Q8_0 first.
