Most users choosing between full-precision flagship LLMs and light-weight local options trade accuracy for cost and latency. This Hugging Face release narrows that gap by combining a targeted "heretic" finetune (reduced refusal behavior) with metric-calibrated NEO-CODE Di‑Matrix GGUF quantization so that several quant variants approach BF16 performance while greatly reducing model size and inference cost.
Key Capabilities
- Quant-aware variants: Multiple GGUF quants (IQ2_M, IQ3_M, IQ4_xs/NL, Q4_K_S/M, Q6_K, Q8_0) benchmarked against BF16 with reported Same-Top-P and KLD metrics (e.g., Q4 variants ~94% same-top-p; IQ2_M ≈83% of BF16 by their metrics), enabling trade-offs between speed, memory and fidelity.
- Uncensored finetune: A post‑processing "Heretic" finetune reduces refusal rates and shifts behavior for creative and open responses (benchmarks in model card show large differences vs. original in refusal counts and KLD measures).
- Multimodal/vision support: Includes a small mmproj visual processor file for image-capable runs; suggested sampling and mode presets provided for "thinking" vs. instruct modes.
- Deployment-friendly: Packaged as Hugging Face Transformers artifacts and GGUF files with suggested serving integrations (vLLM, SGLang, transformers serve) and sampling recommendations for coding vs. creative tasks.
Who it's for and tradeoffs
Great fit if you: want a locally runnable Qwen3.6‑27B variant that preserves most BF16 behavior while saving memory and enabling long-context (>8k–256k) or multimodal experiments; need a model tuned toward fewer refusals and creative/coding outputs; or plan to serve on vLLM/sglang with GGUF quant files.
Look elsewhere if you: require strict safety, provenance, or filtered behavior (this is explicitly an "uncensored/heretic" build), need official vendor support, or demand reproducible results tied to an upstream, unmodified Qwen3.6 release. Quantized performance depends on the chosen quant and hardware—expect small but measurable distributional shifts (KLD/RMS Δp) versus BF16.
Where it fits
This artifact sits between full‑precision Qwen3.6 releases and lightweight community quant builds: it prioritizes metric-backed quant engineering (Di‑Matrix merging, tensor tweaks) to maximize fidelity per byte. Use it for local inference, experimentation with quant trade-offs, or as a base for further fine‑tuning—avoiding it when certified safety, vendor guarantees, or strict content filters are required.
