Most video LoRAs condition via text or separate reference blocks; this project takes a different tack by teaching the model a pixel-to-pixel mapping from a near-clean encoded source clip to a noised target timeline. That design makes sharpening deterministic and positionally cheap, so the adapter produces crisp, photorealistic detail without asking the transformer to search for correspondences.
What Sets It Apart
- Guide-latent conditioning: the source clip is encoded by the video VAE and inserted as a latents-aligned guide on the target timeline, so guide token (t,y,x) aligns directly with target token (t,y,x). This removes the positional search cost and yields pixel-level correspondence rather than a textual paraphrase.
- Training regime: the guide was kept near-clean (~0.1% noise) while targets were noised normally, so the LoRA learns to map a clean source to a noisy target (useful for sharpening and restoring detail).
- Practical integration: shipped as a rank-64 LoRA primarily trained against ref2va workflows; includes a ready ComfyUI workflow and comparison videos. Uses ComfyUI's native MiniMaxH3AddGuide (frame_idx = 0) for guide input; an optional ai-toolkit node is provided to resize clips for the guide.
Who it's for and tradeoffs
Great fit if you use MiniMax H3 (ref2va) in ComfyUI and want a simple second-pass to sharpen or enhance an existing clip while preserving photorealism. It works best when the source guide is the same resolution as the target, anchored at frame 0, and both clip lengths fall on the model's valid frame lengths (the model expects clip lengths in the sequence 5, 22, 39, 56, 73, 90, 107, 124, … — i.e. 17k+5 pattern). Look elsewhere if you need a LoRA tuned for image-to-video fl2va workflows (this LoRA is less tested there), or if you cannot provide tightly aligned guide latents at the same resolution and valid clip length. Combining this guide-latent conditioning with the native ReferenceToVideo (which uses an appended reference clock) is possible but may produce different behavior since it feeds two conditioning styles.
Where it fits
Use this adapter as a sharpening/refinement pass in a ref2va pipeline or to enhance already-generated clips. It is primarily a refinement tool rather than a single-step generator from scratch, although it can be applied directly with the understanding that behavior differs from pure generation LoRAs.
