Fitting a model this large onto one accelerator requires more than low-bit quantization: this release also removes a quarter of the routed experts. That combination brings the file down to 179.9 GB, small enough for a single B200, but its benchmark scores should be read as results for this compressed variant—not as a direct measure of the full GLM-5.3 model.
Key Capabilities
- Reduced expert count: Neural architecture search selected 192 of the original 256 routed experts in each MoE layer. This cuts the parameter count from 753B to 563B without retraining.
- Compact GGUF weights: The model retains Unsloth’s UD-IQ2_M quantization, averaging 2.56 bits per weight, and is 24.6% smaller than the 256-expert version. On a B200, the remaining memory can support roughly 100K tokens of shared f16 KV cache.
- Reported evaluation results: The release reports 92.7% on serial HumanEval pass@1 and 84.8% on GPQA-Diamond. These are locally measured results from the project’s own setup and are not directly comparable with the official GLM-5.3 benchmarks.
- Local runtime focus: It is packaged for llama.cpp; the MTP layer is removed, so MTP speculative decoding is not supported.
Who It’s For—and the Trade-Off
A good fit for users with a B200, or a Mac Studio with at least 192 GB of unified memory using mlx-serve, who want to run a GLM-5.3-class model locally. Look elsewhere if you need the full expert set, reliable top-tier Chinese knowledge performance, or MTP decoding: expert reduction and very low-bit weights compound quality loss, and the project notes particular concern about Chinese knowledge QA.
