Why this matters now
Efficient on-device reasoning is shifting from a niche to a practical requirement for researchers and edge deployments. This preview targets that shift by trading conventional dense capacity for a ternary-weight MoE design and engineering optimizations to move the reasoning vs. memory/speed Pareto frontier.
Key Capabilities
- High on-device throughput and compact footprint — small checkpoint (≈5.3 GB) and reported 200+ tokens/sec on an M4 Mac mini, which means faster iteration and lower hardware cost for local evaluation.
- Strong math and logic reasoning for its class — benchmarked against contest-style datasets (IMO/AIME/HMMT-style tasks) and positioned to outperform many dense models at similar memory points, so it’s useful for hard reasoning evaluation and research.
- Very long context support — 131,072-token context enables long-form chains of thought and large-document problem statements, so you can evaluate extended reasoning and retrieval-augmented prompts without truncation.
- Preview-quality trade-offs — focused on raw reasoning rather than agentic behaviors or extensive RL fine-tuning, so practical interactive agents or broad conversational tasks may underperform compared to generalist LLMs.
Who It's For and Trade-offs
Great fit if you are: researchers benchmarking reasoning per-cost, engineers prototyping local inference on Apple Silicon or constrained GPUs, or anyone comparing MoE/ternary designs for efficient LLMs. Look elsewhere if you need: production-hardened conversational agents, extensive safety/mitigation measures out of the box, or broad instruction-following that requires more post-training or RLHF.
Where It Fits
Technically positioned as an efficiency-first reasoning model: it emphasizes throughput, long context, and contest-style problem performance rather than being a general-purpose assistant. Treat it as a research/edge inference artifact to test new deployment and evaluation workflows.
Architecture Notes
Preview implements a 24-layer MoE with 256 experts (8 active) and a ternary-weight format plus a 3:1 SWA-512:GA attention pattern. The Transformers implementation depends on Triton and FlashAttention for CUDA environments; a separate on-device runtime is used for Apple Silicon measurements. Continued training and broader fine-tuning are planned before a full release.
