Tag

GGUF-format QAT (quantization-aware training) build of Gemma 4 12B that reduces memory needs for local or lightweight inference while preserving near bfloat16 quality. Ready for any-to-any conversational pipelines and ecosystem deployment.

A GGUF release of Gemma 4 26B A4B (QAT) packaged by Unsloth for local multimodal inference — quantization-aware trained to keep near-bfloat16 quality while significantly lowering memory requirements, compatible with Transformers and Unsloth tooling.

Implements MXFP4 quantization on MoE experts plus a BF16 DFlash block-diffusion drafter to propose whole-token blocks for verification, cutting memory bandwidth and backbone forward passes for trillion‑parameter text generation—targeting long‑context, agent and code workloads.

Generates text from interleaved text, image, and short-video inputs using discrete diffusion and block‑autoregressive multi‑canvas sampling; built on a sparse MoE (8/128) Gemma 4 backbone and optimized for low‑latency inference and very long contexts (up to 256K tokens).

A community-distributed GGUF bundle of Google DeepMind’s DiffusionGemma (26B A4B) with multiple quantization variants for local image-text-to-text inference. Targets experimentation and offline deployment via the DiffusionGemma llama.cpp branch and llama-diffusion-cli; choose quantization for GPU memory vs. fidelity trade-offs.

A quantized 27B coder LLM fine-tuned for repository-level code generation, multi-turn tool calling, and agentic workflows — packaged for local GGUF/llama.cpp deployment with MTP speculative decoding and trace-inversion SFT. Optimized for developer tooling; experimental and not fully safety-validated.

Implements a blockwise sparse attention (MiniMax Sparse Attention) that scores and Top-k selects key-value blocks per Grouped Query Attention group to enable attention over million-token contexts. Paired with an exp-free Top-k GPU kernel and KV-outer sparse execution, it reduces per-token attention compute and yields large prefill/decoding speedups.

An agentic multimodal coding model for long-horizon software tasks: MoE architecture (1T params, 32B activated), 256K context, image/video input, native int4 quantization and preserved chain-of-thought (thinking) mode. Tuned for multi-step coding workflows and vLLM/SGLang deployment.

NVIDIA’s open-source library that compiles Transformer blocks into highly-optimized TensorRT engines for blazing-fast LLM inference on NVIDIA GPUs.

Pythonic framework to inject experimental KV-cache optimizations into HuggingFace Transformers stacks.

NVIDIA Dynamo is an open-source, high-throughput, low-latency inference framework that scales generative-AI and reasoning models across large, multi-node GPU clusters.