Tag

SadTalker

Generate a lip-synced talking-head video from a single portrait image and an audio clip using learned 3D motion coefficients for realistic expression and head motion. Offers still/reference modes, Colab/HuggingFace demos, and an Apache-2.0 license.

FunASR

Bundles ASR, voice activity detection, punctuation, and speaker diarization into one pipeline, with pretrained models like Paraformer and SenseVoice. SenseVoice runs ~17x realtime on CPU; also ships streaming ASR and an OpenAI-compatible API.

SD.Next

Web and desktop/mobile WebUI for generating, editing, captioning and processing images and videos with Stable Diffusion and many diffusion models. Key features include automatic model download, SDNQ on-the-fly quantization for VRAM savings, balanced CPU/GPU offload, multi-backend GPU support, and built-in captioning/tagging/upscaling workflows.

nanoGPT

Reproduces GPT-2 (124M) from scratch on OpenWebText in ~4 days on an 8xA100 node, with the whole stack kept to two ~300-line files: train.py for the loop and model.py for the architecture. A char-level Shakespeare run finishes in ~3 minutes on one GPU.

X-AnyLabeling

X-AnyLabeling is a powerful annotation tool integrated with an AI engine for fast and automatic labeling. Designed for multi-modal data engineers, it offers industrial-grade solutions for complex tasks. Supports images and videos, GPU acceleration, custom models, one-click inference for all task images, and import/export formats like COCO, VOC, YOLO. Handles classification, detection, segmentation, captioning, rotation, tracking, estimation, OCR, VQA, grounding, etc., with various annotation styles including polygons, rectangles, rotated boxes.

NVIDIA PhysicsNeMo

Modular PyTorch-based framework for building, training, and deploying physics-informed ML models (neural operators, PINNs, GNNs, diffusion). Provides GPU‑optimized training, domain-specific datapipes for meshes/point clouds, distributed scaling and a model zoo.

vLLM

Open-source LLM inference and serving engine built around PagedAttention, which manages the KV cache like OS virtual memory to cut waste and raise throughput. Supports continuous batching, KV cache sharing, quantization, and an OpenAI-compatible API.

Retrieval-based-Voice-Conversion-WebUI

Web UI to train and run retrieval-based voice conversion models from small datasets (≈10 minutes), featuring top-1 feature retrieval to avoid timbre leakage, model fusion, real-time conversion, vocal separation, and multi-hardware support.

Axolotl

Streamlines post-training and fine-tuning for large language and multimodal models with a single YAML-driven pipeline. Supports LoRA/QLoRA, full fine-tuning, preference tuning, RL methods, multi-GPU/FSDP/DeepSpeed, and many model backends (Hugging Face, local checkpoints).

LLaVA: Large Language and Vision Assistant

Connects a frozen vision encoder to a language model via visual instruction tuning, yielding an open multimodal assistant that follows image-grounded instructions. Released checkpoints span 7B-34B and approach GPT-4V on vision-language benchmarks.

LitGPT

Provides from‑scratch implementations and end‑to‑end workflows to pretrain, finetune, evaluate, and deploy 20+ LLMs with reproducible YAML recipes, multi‑GPU/TPU support, quantization, and LoRA/QLoRA options.

Massively Multilingual Speech (MMS) - 300m

Pretrained multilingual speech encoder for ASR and related speech tasks (300M parameters), trained with wav2vec2-style self-supervision on ~500k hours across 1,400+ languages. Designed for 16 kHz audio and typically fine-tuned for downstream ASR/translation/classification.