Local inference is where compact quant files deliver practical access to large-model reasoning. This GGUF release packages a merged Qwen3.6-35B checkpoint that was supervised-fine-tuned (LoRA → merged) on Claude Opus 4.6-style chain-of-thought data, then exported into multiple GGUF quantizations so you can run reasoning-focused generations on llama.cpp-compatible runtimes without needing the original training stack.
Key Capabilities
- Reasoning-focused SFT behavior: the source checkpoint was distilled from Claude Opus 4.6-style reasoning trajectories and reports a large jump on a lightweight MMLU-Pro check (exact_match ~75.71% vs base ~42.86% on the smoke harness used by the author), indicating stronger closed-book reasoning in many prompts.
- Multiple GGUF quant options: Q4_K_M, Q5_K_M, Q6_K, Q8_0 are provided to balance VRAM/RAM constraints and quality trade-offs for local inference with llama.cpp or similar runtimes.
- Practical local deployment: the release targets runtime compatibility (gguf/llama.cpp) rather than continued training; the fine-tune was text-only, so any multimodal behaviors inherit from the Qwen3.6 base but were not improved by this run.
- Training & merge notes: supervised fine-tuning used LoRA (attention-only), then the weights were merged and quantized; reported training used long contexts (up to 32k) and modest final training loss for the run.
Who it's for — and tradeoffs
Great fit if you want offline/text-only inference that emphasizes chain-of-thought style reasoning and you need quantized files that run on llama.cpp-like runtimes. It's useful for researchers and developers benchmarking reasoning SFTs locally, or for deploying a reasoning-oriented model where cloud access is undesirable. Look elsewhere if you need verified per-quant benchmarks (the provided MMLU-Pro score was measured on the merged source checkpoint, and quantization can change results), require image/video fine-tuning (this run is text-only), or need vendor-supported SLAs. Also expect lower quality at the smallest quant levels compared with full-precision checkpoints; always validate with your task and decoding settings.
