GGUF-quantized build of Moonshot AI's Kimi K3 for local inference: MXFP4-aware quantization, image-text-to-text pipeline support, native vision and a 1,048,576-token context window. Intended for local GGUF runtimes (vLLM, SGLang, TokenSpeed) with Kimi K3 license constraints.
Runs Kimi K3 inference on CPU by streaming model weights from storage instead of keeping them all in RAM. The project reports 8.24 GB peak memory, with substantially slower token generation at low-memory settings.
Provides 997 chain-of-thought cybersecurity reasoning records distilled from the Kimi K3 model, each with an explicit <think> trace and a technical resolution or structured tool invocation. Includes verified tool-call objects, diffs, cross-domain coverage, and token-level metadata for fine-tuning and evaluating reasoning models.
A multimodal model for coding, visual understanding, and multi-step tool use, with a 1-million-token context window. Its MoE architecture has 2.8T total parameters and activates 104B per token; weights are open, but local inference demands substantial compute.