Most production translation stacks must trade latency, model size, and the ability to follow translation instructions. Hy‑MT2 narrows that trade-off by shipping a family of models designed for "fast-thinking" translation: a compact 1.8B variant for on-device and low-latency scenarios, plus 7B and MoE 30B-A3B variants for higher-quality outputs and instruction-following.
Key Capabilities
- Multilingual instruction following: supports translation among 33 languages and can follow translation-style, terminology, and formatting instructions for structured or free-form inputs.
- Small-model on-device option: AngelSlim 1.25-bit quantization reduces the 1.8B model to roughly 440 MB and improves inference speed (~1.5× reported), enabling deployment on constrained hardware.
- Scalable quality/performance tiers: 7B and 30B-A3B models offer higher accuracy and robustness for domain-specific or business translation tasks, while the 1.8B focuses on cost and latency.
- Evaluation and ecosystem: the release includes IFMTBench (an instruction-following translation benchmark) and integration examples for transformers, vLLM, and llama.cpp/gguf formats to ease adoption.
Who it's for and tradeoffs
Great fit if you need a multilingual translation model that can be deployed across a range of environments — from on-device mobile/embedded inference to server-side, higher-capacity instances — and if instruction-following (terminology, style, structured-data preservation) matters. Look elsewhere if you require open datasets or permissive licensing details not provided with this release, or if you need models pre-integrated with a specific commercial API ecosystem; larger commercial MT systems may still outperform on some narrow, high-resource language pairs or specialized SLT benchmarks.
Where it fits
Hy‑MT2 sits between tiny translation models intended only for basic bilingual tasks and very large commercial models: it offers a pragmatic middle path (compact 1.8B with aggressive quantization vs. 7B/30B for accuracy) and positions itself as an accessible, research-backed option for teams who want to run multilingual translation at varying cost/latency points. The authors also highlight participation in WMT26 tasks and published evaluations (see the model card and report) for benchmark comparisons.
