Why this matters
Open-source 9B models that combine strong instruction-following, multimodal input and practical local quantization are rare. This release packages a multi-stage fine-tune of Qwen3.5‑9B with NEO IMATRIX GGUF and an MTP (multi-token prediction) variant, targeting users who need a compact but high-performing LLM for local inference with long-context and vision support.
What sets it apart
- Multi-stage, multi-model fine-tune and “heretic” (uncensored) tuning — the author states the model was merged and tuned from several 9B fine-tunes to raise instruction-following and reasoning ability; so what? it aims to push 9B-class utility closer to larger checkpoints for many tasks.
- NEO IMATRIX GGUF + MTP quants — the release includes both regular and MTP (fast) GGUFs; so what? IMATRIX quants report 2–4% accuracy gains over standard quants and MTP can increase token throughput (author reports ~130 t/s for regular vs >185 t/s for MTP on specific hardware) when multi-token acceptance is high.
- Native long context + vision — 256k native context (vision enabled via a separate mmproj) lets users run extended reasoning and image-text-to-text jobs locally; so what? it’s suited for long-form summarization, document/VQA workflows and multimodal agents without immediate cloud dependence.
- Benchmarks & practical tuning guidance — the model card reports evaluation snapshots (example: bf16 ARC-C ~0.649, ARC-E ~0.832, BoolQ ~0.895 in reported runs) and provides recommended sampling parameters and smoothing settings for common UIs; so what? you'll get actionable settings to reproduce generations and test quants.
Who it’s for and tradeoffs
Great fit if you want a locally runnable, 9B-class Qwen3.5 variant that: (a) prioritizes instruction-following and creative/roleplay generations, (b) needs long-context multimodal capability, and (c) wants both regular and MTP GGUF options for speed/throughput tradeoffs. Look elsewhere if you require a safety‑filtered, refusal‑strict model (this is explicitly released as uncensored/heretic), need vendor-certified production SLAs, or cannot accept the legal/ethical implications of an uncensored model in your deployment. Additional practical notes: vision requires the provided mmproj file, and IMATRIX quants may need specific engine support (some AI apps require llama_HF config adjustments). License: Apache-2.0.
Where it fits
This model aims to occupy the niche between lightweight local LLMs and much larger server-side models: offering a 9B footprint with benchmark-driven tuning to close some performance gaps versus 27B+ models on selected tasks, while providing quantized GGUF artifacts for end users who run inference on consumer/prosumer GPUs.
Quick usage notes (high-level)
- Try both regular and MTP GGUFs to see which gives better token acceptance on your hardware; MTP benefits appear when multi-token acceptance is >50%. Keep temperature and repetition-penalty settings consistent with the author’s suggestions for “thinking” vs “instruct” modes. For vision, place the mmproj alongside the GGUF to enable image inputs. Be mindful of safety and compliance when using an uncensored model in public-facing or regulated products.
