Tag

GLM-5.2 (Unsloth GGUF)

Provides GGUF-quantized GLM-5.2 builds for local text-generation with a solid 1M-token context, dynamic 1-/2-bit quant options, and Unsloth runtime integrations — targeted at long-horizon coding, reasoning and agent workflows. MIT licensed.

TurboServe: Serving Streaming Video Generation Efficiently and Economically

Serves interactive, long-lived streaming video-generation sessions by jointly scheduling session placement and GPU autoscaling to meet tight per-chunk latency. Combines migration-aware placement, load-driven autoscaling, coalesced chunk processing, GPU–CPU offloading and NCCL GPU–GPU migration; reports ~37% reductions in worst-case per-chunk latency and GPU operating cost.

Qwythos-9B

A 9B reasoning LLM fine-tuned from Qwen3.5 that ships with a 1,048,576-token context, native function-calling and tool-use, and notable benchmark gains (+34 MMLU, +30 gsm8k-strict).

NVIDIA GLM-5.2 NVFP4

Provides a pre-quantized NVFP4 checkpoint of GLM-5.2 for long-context reasoning and coding; reduces model footprint so GLM-5.2 can run on multi‑GPU Blackwell nodes and is ready for inference with SGLang and vLLM.

nvidia/Qwen3.6-27B-NVFP4

NVFP4-quantized variant of Qwen3.6-27B that reduces parameter bits from 16 to 4, cutting disk and GPU memory requirements by ~2.5× while keeping comparable benchmark accuracy; ready for vLLM-based inference on NVIDIA hardware and supports long, multimodal contexts.

LFM2.5-230M

230M-parameter multilingual instruction-tuned text-only LLM for on-device agentic pipelines and data extraction; 32K context, 19T-token pretraining, optimized for fast CPU/edge inference (e.g., 213 tok/s on Galaxy S25 Ultra, 42 tok/s on Raspberry Pi 5); not for heavy reasoning or complex code generation.

Inflect-Micro-v2

Local English text-to-waveform TTS producing a single fixed synthetic voice in a deployable package below 10M parameters. Offers deterministic seeds, punctuation-aware long-text chunking, CPU/CUDA and ONNX runtime options, measured evaluations and a compact FP32 footprint; English-only, one voice.

Inflect-Nano-v2

Generates English speech locally from text into 24 kHz waveforms with a fixed synthetic male voice. Complete text-to-waveform TTS under ~4M parameters (≈16 MB FP32), supports CPU/CUDA inference, deterministic seeds, long-text chunking and an ONNX export path under Apache-2.0 license.

Ornith-1.0-35B-GGUF

A self-improving, agentic coding LLM tailored for terminal-style coding agents and tool-calling, provided as 35B MoE GGUF weights with very large context support. Trained with reinforcement learning to jointly generate task scaffolds and solutions; designed for local inference and OpenAI-compatible tool endpoints.

DeepSeek-V4-Pro-DSpark

Mixture-of-Experts LLM designed for million-token contexts, combining hybrid compressed attention, FP4/FP8 quantization-aware training for MoE experts, and multi-mode 'thinking' (Non-think/Think High/Think Max); includes a speculative-decoding extension for faster inference.

The Mirage of Optimizing Training Policies: Monotonic Inference Policies as the Real Objective for LLM Reinforcement Learning

Proposes Monotonic Inference Policy Improvement (MIPI) and a two-step Monotonic Inference Policy Update (MIPU) to address training–inference probability mismatch in LLM reinforcement learning by constructing sampler-referenced candidate updates and accepting synchronized updates using an inference-gap proxy; shows improved reasoning accuracy and stability under FP8-quantized rollouts.

Huihui-GLM-5.2-abliterated-GGUF

An uncensored GGUF build of GLM-5.2 that applies weight “abliteration” to remove refusal filters and produce a locally runnable text-generation model; includes quantization conversions and shard-merge instructions, intended for experimental research rather than production use.