Provides portable tensor operations and low-level compute kernels for machine learning. Its backend support and low-bit quantization help developers build lightweight inference software across varied hardware.
Curated free roadmap and runnable Colab notebooks to learn Large Language Models, split into LLM Fundamentals, LLM Scientist, and LLM Engineer tracks. Includes hands-on labs for quantization, model merging, fine‑tuning, retrieval augmentation, and local/server/edge deployment workflows.
Provides a family of multilingual translation models (1.8B, 7B, 30B-A3B) supporting 33 languages and instruction-following; offers low-bit quantized formats (1.25-bit, 2-bit, FP8, GGUF) and deployment guides for on-device and server inference.
Runs large open-weight MoE models locally by coordinating CPU and GPU resources, caching experts, and reallocating VRAM at runtime. It targets consumer hardware, from laptops to workstation GPUs.
Rewrites AI-generated English and Chinese drafts so they read like human writing while preserving every number, date, unit, name and quote. Runs locally with multiple GGUF quantized builds and a strict byte-for-byte prompt format for consistent rewrites.
Provides quantized Qwen-Image-2.1-Turbo weights for text-to-image generation across GGUF, Safetensors, and MLX formats. It retains the model’s 8-step Turbo workflow, with options for different hardware and runtimes.
Provides compact mixed-precision GGUF quantizations of UkisAI's Swift 1.5 (derived from Qwen3.8-27B), using ISTA GSQ-RCO per-tensor allocations with Swift-specific refinement. Offers multiple 8–12 GB tiers, optional MTP heads, and KLD evaluation against the Swift BF16 baseline.
Capability-targeted compression of Qwen3.8-Flash-Next: half the experts are removed and remaining weights quantized to 3.5 bpw, producing a 58.4 GB GGUF (29.6 GB resident) that preserves coding and multimodal ability while trading off other domains.
Provides a high-fidelity mixed-precision (≈3-bit) GGUF quant of Qwen3.8-27B tailored for long-horizon agent workloads and cyber-focused red-teaming. Preserves reasoning, thinking mode, MTP speculative decoding and vision (via a separate mmproj); released as an uncensored/abliterated research build under Apache-2.0.
Transcribes English speech into punctuated, capitalized text — a 164 MB quantized ASR model that averages 5.21% WER across seven Open ASR Leaderboard sets. Optimized for on-device and CPU/GPU inference, with fast runtimes on Apple M5 and Docker/GPU support.
Compresses recurrent states in linear-attention LLMs via spatial–temporal post-training quantization, allocating bits by error lifetime and per-row impact to preserve accuracy (6-bit ≈ FP32) while cutting serving memory up to 68.7%.
On-device speech-to-text for short clips (up to 30s) in seven languages, yielding transcripts, word-level timestamps and per-frame speech embeddings in a single 16.9 MB model. Runs on the Needle CPU engine with 2–4 bit quantization, supports keyword biasing and returns empty transcripts for silence.