Beam shows a pragmatic tradeoff for teams that want frontier-scale capacity without paying full inference cost: by routing each token to 8 of 256 experts it holds 501B parameters in memory while computing with ~23B active parameters per token. That design enables sustained long-horizon reasoning (1M-token window) and agentic tool workflows while keeping per-token compute closer to mid‑sized models.
Key Capabilities
- Sparse MoE architecture and scale: 52 layers (3 dense + 49 MoE), 256 experts, 8 experts routed per token → 501.3B total params, ~23.4B active params. This means large representational capacity is available without full dense compute on every token.
- Long-context and attention design: hybrid sliding-window + global attention (most layers use sliding windows, every sixth and final layer attend globally) to support 1,048,576-token contexts with manageable compute patterns.
- Practical checkpoints & numerics: official checkpoints in BF16, FP8 and NVFP4; BF16 full checkpoint is ~1 TB, NVFP4 reduces disk size (~280–294 GB) for Blackwell-class GPUs while preserving much of BF16 quality.
- Agentic and engineering focus: instruction-tuned for reasoning, code, and tool use; includes Harmony-format chat/tool grammar, speculative decoding support, and day‑0 integrations for transformers, vLLM, SGLang plus a Rust-serving harness.
- Evaluation profile: benchmarked across coding/agentic and reasoning suites (e.g., strong scores on long-horizon and agentic coding benchmarks) and run through safety evaluations with high refusal rates on harmful requests.
Who it's for + trade-offs
Great fit if you need a locally controllable, license-permissive model with frontier capacity for coding, long-horizon agents, or research that benefits from long context and MoE capacity. It’s suitable for teams that can provision large aggregate GPU memory (all experts kept in memory) and want to tune or run full-parameter RL/SFT. Look elsewhere if you lack the hardware footprint (serving requires memory comparable to a 501B model even when running at 23B compute), need a pure small-model latency envelope on commodity GPUs, or require native multimodal inputs (Beam is text-only).
