AIAny
AI Model2026
Icon for item

Qwen3.8-27B-Uncensored-GGUF

Provides locally runnable GGUF quantizations of Qwen3.8-27B with the MTP speculative-draft head preserved and a Heretic weight edit that substantially reduces refusal rate. Ships multiple quant sizes with published imatrix and perplexity measurements for local inference under Apache‑2.0.

Introduction

Why this matters

This release makes a deployable, locally runnable variant of Qwen3.8-27B that keeps the model's native MTP speculative-draft head and applies a targeted weight edit to reduce safety-driven refusals. That combination preserves the speculative-decoding speedups while exposing a measured reduction in refusal behaviour, so operators can evaluate trade-offs between safety constraints and capability in a reproducible way.

Key Capabilities
  • MTP preserved: the multi-token-prediction (MTP/NextN) draft head is grafted back into each shipped GGUF, enabling llama.cpp speculative decoding without a separate sidecar.
  • Measured behaviour: refusal rate on a 100-item harmful prompt set dropped from 98/100 (base) to 12/100 for the published point; KL divergence vs base (first-token) is 0.1191. Benchmarks (0-shot) show a mean delta of −0.5 across MMLU, ARC-Challenge, HellaSwag and Winogrande under the test harness used.
  • Multiple quantizations: files range from IQ2_M (~10.6 GB) up to Q8_0 (~29.0 GB), with published per-quant perplexities on wikitext-2 and an imatrix calibration artifact to reproduce quants.
  • Speculative decoding supported and benchmarked: fused and noMTP/draft split workflows are provided, with throughput measurements and guidance for llama.cpp flags.
Method and verification
  • Refusal edits were produced by Heretic, which co-optimizes refusal count and KL divergence; edits were applied at bf16 as a LoRA that was merged into the base before conversion.
  • The MTP tensors are copied verbatim from the base checkpoint after the merge; every GGUF was inspected post-quantization to confirm the MTP block survives (65/65 blocks reported for fused files).
  • Calibration imatrix was computed from the f16 GGUF (published imatrix included) and used to build the mixed-precision quants; the card documents procedures and the exact llama.cpp commits used.
Who it's for & trade-offs

Great fit if you need a local, reproducible deployment of Qwen3.8 that: wants MTP speculative decoding, needs multiple quant size options for tight VRAM targets, and is prepared to evaluate and accept reduced refusal behaviour under a specified safety trade-off. Look elsewhere if you require fully enforced refusal/safety guarantees, extensive generative-code/math evaluations (not provided here), or if you cannot run a recent llama.cpp build that supports MTP. Lower-bit quants (notably IQ2_M) show larger degradation and greater instability around prior refusal boundaries; evaluate on Q6_K or Q8_0 first.

Information

Categories

More Items

Hugging Face
AI Model2026

An experimental Qwen3.8-27B variant edited in weight space to use fewer reasoning tokens; reported evaluations show higher accuracy on GPQA Diamond and LiveCodeBench v6.

Hugging Face
AI Model2026

Listens and responds during live speech, interpreting cues such as hesitation, emphasis and sarcasm. Its replies can include expressive nonverbal sounds; Tag, Flash and E2E builds trade expressiveness, throughput and first-audio latency.

Hugging Face
AI Model2026

Processes incoming and outgoing speech in parallel for real-time conversation, producing synchronized audio and text at 7.5 frames per second. Its continuous-audio design targets natural turn-taking rather than a conventional voice-assistant workflow.