On-device translation is often constrained by model size and runtime; Hy-MT1.5-1.8B-1.25bit demonstrates that a commercial-grade translation model can be compressed to a phone-friendly footprint with little accuracy loss. By combining the HY‑MT1.5 translation family with the Sherry 1.25-bit quantization and mobile-optimized kernels, this release aims to make high-quality offline translation practical on ordinary smartphones.
What Sets It Apart
- Sherry 1.25-bit fine-grained sparsity: packs weights with a 3:4 sparsity scheme (three 1-bit values + one zero per 4 weights), yielding an effective 1.25-bit width and shrinking a ~3.3GB FP16 model to ~440MB — so what: enables truly offline models that previously required gigabytes of storage.
- Translation-trained foundation: built on HY‑MT1.5 (a translation-specialized 1.8B model) and evaluated on Flores-style benchmarks — so what: achieves competitive translation quality vs much larger open and commercial models while staying compact.
- Mobile deployment engineering: includes STQ kernel alignment and an Android demo — so what: the model is packaged and benchmarked for real-device latency/memory constraints rather than only server-side evaluation.
- Multi-directional coverage: supports 33 languages and 1,056 translation directions — so what: useful for multi-lingual offline workflows where server access is unavailable or undesirable.
Who It's For (and trade-offs)
Great fit if you need high-quality, offline translation on limited-memory devices (researchers building mobile translators, app developers shipping offline multilingual features, privacy-sensitive deployments). Look elsewhere if your priority is absolute top-tier BLEU/ChrF on narrow high-resource language pairs (cloud-hosted 32B+ models or specialized server ensembles still outperform in head-to-head quality) or if you require full FP16 precision for downstream fine-tuning.
Where It Fits
Positioned between tiny on-device NMT and large cloud translators: it trades modest parameter count (1.8B) + aggressive quantization for a strong offline quality/size balance. Compared to cloud APIs, it removes latency, cost-per-query, and data-exit concerns; compared to larger local models, it targets much lower storage and RAM budgets.
How It Works (brief)
The core compression uses Sherry, a hardware-aware ternary/fine-grained sparsification scheme that compresses groups of weights into a compact bit-layout (effective 1.25-bit). Practical on-device speed comes from pairing the quantized weights with a custom STQ kernel optimized for mobile SIMD instructions. The release also provides GGUF and an Android demo for direct experimentation.
Overall, this model is a pragmatic engineering combination of translation-specialized training and extreme quantization that makes high-coverage multilingual offline translation feasible on typical phones — with honest trade-offs in precision vs. server-grade models.
