Long reasoning traces drive up latency, compute and output-cap failures; Swift addresses this by discouraging specific "reasoning‑marker" tokens during fine‑tuning so Qwen3.8‑27B produces much shorter reasoning rollouts with minimal accuracy loss. The GGUF release packages that adapter into an F16 conversion ready for llama.cpp, Ollama, LM Studio and similar runtimes, and includes a multimodal projector and multiple quant tiers for deployment trade‑offs.
Key Capabilities
- Measured token efficiency: median thinking/token reductions up to ~58.3% on GPQA and large mean reductions across MMLU, IFBench, ERQA and math benchmarks, yielding roughly 1.95x speedups on several tasks while keeping accuracy within ~1% of the BF16 base. This typically lowers output‑cap truncation and end‑to‑end latency for long reasoning runs.
- GGUF tiers and sizes: F16 GGUF shards (three text shards ~54.7 GB total) plus a ~0.9 GB multimodal projector; multiple quant tiers (Q8_0, Q6_K, Q5_K_M, Q4_K_M, and smaller experimental IQ/Q tiers) let you choose fidelity vs memory (Q8_0 ≈29.1 GB, Q4_K_M ≈18.0 GB, etc.).
- Deployment features: built to run with llama.cpp (including llama-server), Ollama, LM Studio and Jan AI; includes MTP (Multi‑Token Prediction) layers for speculative decoding and an embedded chat template that separates reasoning from final answers. A hosted OpenAI‑compatible API (model id
swift) is available from UkisAI and free for research use without an API key. - Multimodal support: a packaged mmproj projector enables image input when used with supported runtimes. MTP layers (stored at Q8_0) support faster speculative decoding when enabled.
Who it's for and trade‑offs
Great fit if you need to run long, structured reasoning or agentic tasks locally and want lower memory/latency without re‑engineering prompts: Swift substantially reduces internal reasoning token inflation and makes long traces tractable on quantized runtimes. It is also useful when you want an immediate llama.cpp/Ollama workflow with preconfigured sampling and reasoning templates. Look elsewhere if you require absolute bit‑for‑bit parity with the original BF16 checkpoint for every long‑context edge case or if your deployment requires an unconstrained commercial license above the Swift Open License threshold (organizations with ARR > US$1,000,000 need a separate enterprise license). Note also that the GGUF conversion passed smoke and finite‑tensor checks, but the full multimodal/benchmark suite was not re‑run on the GGUF files; some INT4 evaluation numbers come from other checkpoints (BF16/INT4) rather than the F16 GGUF build.
Where it fits
Swift occupies the middle ground between high‑fidelity BF16 checkpoints and ultra‑small INT4 quants: choose Q8_0 for maximum fidelity, Q6_K for long agentic runs needing stricter formatting, and Q4_K_M (≈18 GB) for everyday 24 GB GPU cards where memory and speed matter. Its design prioritizes shorter, less error‑prone reasoning traces rather than maximizing raw generation diversity.
Implementation notes (short)
The adapter was trained by penalizing tokens identified as reasoning markers that tend to trigger overthinking, and it also includes transfer components from prior ThinkingCap work. The GGUF release is an F16 conversion (not a 4‑bit quant) and bundles the Swift adapter merged into weights; separate quant tiers are provided for deployment. KV cache is small for this hybrid recurrent architecture (approx. 64 KiB per token), which reduces memory pressure on long contexts compared with typical full‑attention 27B layouts.
