Why this matters
Running the full Qwen-Image-2.1 pipeline outside CUDA ecosystems has been frictioned by large bf16 checkpoints and incompatible loaders. This repo packages a refusal‑ablated text encoder in GGUF (Q4_K_M) together with a separate f16 vision mmproj so users can run the same multimodal text→image and edit flows on llama.cpp/ComfyUI (including Mac) with a much smaller disk and memory footprint.
What Sets It Apart
- Refusal ablation (Heretic): directional ablation targets projection tensors to remove many refusal behaviors from the encoder family. Measured family stats: 5/100 refusals, KL divergence 0.0220; independent checks reported 0/20 refusals on held‑out harmful prompts while retaining benign answers.
- GGUF + mmproj packaging for llama.cpp/ComfyUI: language tower quantized as Q4_K_M (≈5.0 GB) plus a separate f16 vision mmproj (≈1.2 GB) so the multimodal encoder is loadable and usable on non‑CUDA platforms via CLIPLoaderGGUF + a small ComfyUI patch node.
- Multiple builds offered: FP8 (safetensors), NVFP4 / W4A8 (CUDA-optimized), bf16 (full precision) and GGUF mixed-precision quantizations, allowing trade-offs between VRAM, speed and fidelity.
- Practical loader fixes: the companion ComfyUI-GGUF-Qwen3VL-TE add-on resolves common GGUF loading errors (shape mismatch and missing architecture metadata) so text-to-image and reference-image editing produce outputs consistent with the bf16 encoder up to quantization noise.
Who It's For and Trade-offs
Great fit if you want to run Qwen-Image-2.1 on non-CUDA hardware (Mac/llama.cpp) or in ComfyUI with reduced memory use and you accept quantization noise and a community-maintained fork. The build is useful for experiments, local image generation pipelines, and lower-VRAM GPUs.
Look elsewhere if you need the stock, unmodified safety/refusal behavior or require an officially supported release: this is a community derivative (not affiliated with the upstream Qwen release) with deliberate ablation of refusal behavior. Quantization and ablation both change model outputs slightly—choose bf16 or higher-precision builds when exact fidelity and original safety responses are required.
Practical notes: the smallest usable configuration is the Q4_K_M GGUF file paired with the mmproj vision file; ComfyUI requires two small add-on nodes to load GGUF Qwen3‑VL models reliably. The repo also documents recommended sampler/settings for Qwen-Image pipelines and provides alternative quantization formats for NVIDIA GPUs.
