Tag
Explore by tags
An asynchronous, high-throughput framework for large-scale reinforcement learning and agentic training that scales to 1T+ MoE models and 1000+ GPUs, with native verifiers integration, end-to-end SFT/RL/evals, and Slurm/Kubernetes deployment; requires NVIDIA GPUs.
Trains multi-step LLM agents with reinforcement learning (GRPO) on your own tasks, wrapping existing agent code behind an OpenAI-compatible client. Its RULER mode scores trajectories with an LLM judge, so there's no reward function to hand-write.
Splits LLM inference into separate prefill and decode GPU pools, then routes requests with KV-cache awareness to cut redundant recomputation. Reports up to 30x throughput on DeepSeek-R1 (GB200 NVL72) and works across TensorRT-LLM, vLLM, and SGLang.
Measures generative AI inference performance with token-level metrics (TTFT, inter-token latency), latency, and throughput under realistic traffic patterns. Provides a multiprocess engine, real-time TUI dashboard, extensible plugins, and integrations for telemetry and result uploads, aimed at inference benchmarking and capacity planning.
Interactive terminal-native AI coding agent that generates, edits, and runs code while managing multiple model providers and optional durable execution. Privacy-first design with local model support, round-robin model distribution, and a flexible agent system for custom workflows.
Reimplements the vLLM inference engine from scratch in ~1,200 lines of readable Python, matching its offline throughput on small models. Prefix caching, tensor parallelism, torch.compile, and CUDA graphs are all kept legible.
Model-compression toolkit for large LLMs/VLMs that integrates quantization (FP8/INT4/etc.), speculative decoding, token pruning and deployment hooks—designed for end-to-end performance on single/multi-GPU inference workflows and research-to-prod model optimization.
A collection of ready-to-run Hugging Face Jobs OCR scripts that add a markdown column (or structured JSON) to image datasets, with model switching, layout detection, server-mode serving, and per-model options for table/form extraction.
Coordinates specialized AI agents — developer, browser, document, multimodal — running in parallel on your desktop to automate multi-step work. Runs fully local via Ollama, vLLM, or LM Studio, with built-in MCP tools and human-in-the-loop checkpoints.
Extensible AI coding-agent toolkit offering a terminal-first coding agent CLI, a unified multi-provider LLM API, TUI and web UI libraries, Slack integration, and vLLM pod support—built to prototype and run agent-driven developer workflows.
Generates and edits full-length music locally using a hybrid language-model planner and a diffusion transformer. Provides text- and audio-driven song synthesis plus selective editing (cover generation, repainting, vocal→BGM), metadata control and lightweight LoRA personalization, and runs on consumer GPUs/CPUs.
Runs text-to-speech with instant voice cloning fully on-device, from phones to GPUs. Built on small LLM backbones (120M-360M params) plus a 50Hz neural codec; clones a voice from ~3 seconds of audio across English, Spanish, German, and French.