A model’s willingness to answer and its ability to answer well are different things. This repository makes a Heretic-modified Gemma 4 31B available in GGUF form for local inference; its lower refusal rate is a behavioral change, not evidence of greater capability or safer output.
Key Capabilities
- Offers quantizations from Q2_K to Q8_0, ranging from about 12 GB to 32.7 GB, so users can trade memory requirements against fidelity.
- Includes separate multimodal projector files for image input; these supplements are needed for image workflows and add roughly 0.9–1.3 GB.
- The upstream model was modified with Heretic’s refusal-direction ablation method rather than trained from scratch. Its model card reports 28 refusals on one 100-prompt test and a 2.18-point average decline across five capability benchmarks; results depend on the test set and evaluation method.
Who It’s For
A good fit for people evaluating or running a locally hosted, less refusal-prone Gemma variant in GGUF-compatible software. Look elsewhere if you need an unmodified assistant, independently validated safety or capability measurements, or a small model: even the smallest listed quant is around 12 GB. A lower refusal rate does not guarantee correct answers or make harmful use appropriate; review the upstream model’s usage terms before deployment.
