Why this matters
High-resolution Minimax H3 video generation is dominated by an expensive VAE decode→pixel-upscale→re-encode round-trip: Minimax H3’s VAE is large (~5B params) and that IO/compute step slows workflows and can introduce ghosting when naive latent interpolation is used. The core insight behind this project is simple and practical: learn to upscale the 24-channel VAE latent directly in latent space (spatial H×W only) so you can generate at low resolution, upscale the latent, then refine at target resolution — cutting time and avoiding interpolation artifacts.
Key Capabilities
- Learned latent upscaling: a neural upscaler trained on ~80,000 paired samples (≈70k video pairs, ≈8k 2K images) maps low-res latents to high-res targets, producing much sharper latent resizes than bilinear/bicubic interpolation.
- 2D and 3D variants: a lightweight 2D ResBlock backbone with temporal layers for speed, and a fully 3D-convolution backbone for stronger temporal coherence. Both preserve the time dimension and only scale spatial axes.
- Flexible output sizing: supported effective scale 1.0×–4.0× (continuous 0.1 steps). The 3D node also supports target pixel dimensions or megapixel targets (aspect-ratio locked and pixel-grid aligned).
- Practical checkpoints & precision options: provided checkpoints in bfloat16/float16 (~691 MB each) and float32 (~1.38 GB) to match GPU/memory tradeoffs.
- ComfyUI integration: companion custom node(s) expose the upscaler as "Minimax H3 Latent Upscaler (2D)" and "(3D)" for straightforward insertion into existing H3 latent workflows.
Who it's for and trade-offs
Great fit if you produce or iterate on Minimax H3 video and want to: generate fast low-res previews, save GPU time on high-res renders, or achieve cleaner temporal coherence than naive latent interpolation. Typical recommended pipeline: generate low-res latent → latent upscaler → re-sample/refine at target resolution → VAE decode.
Look elsewhere if your goal is to post-enhance a finished pixel-space render: this model is not a pixel-space sharpening tool and a direct decode of its output can appear softer unless followed by a refinement/resample step. The 3D variant is heavier on memory/compute than the 2D variant; choose based on your temporal-coherence vs. speed needs.
How it works (brief)
- Input: fully denoised Minimax H3 latent (24 channels), normalized with the same per-channel mean/std used during training.
- Processing: 2D node uses bilinear feature interpolation and temporal 3D-conv inserts; 3D node uses trilinear interpolation and processes spatiotemporal volume jointly.
- Output: upscaled latent (spatial H×W larger, time preserved) that can be decoded or fed into a downstream refine/resample step. The approach avoids the heavy VAE decode/encode cycle and reduces ghosting compared to naive upsampling.
Practical notes: the model is released under Apache-2.0; choose checkpoint precision to match your GPU (bf16/fp16 for Ampere/Ada cards, fp32 for maximum numerical fidelity).
