Merging two strong 9B finetunes into a deeper 64-layer network intentionally trades parameter-count shape for practical VRAM requirements: the result aims to sit between 9B and 27B models in capability while still fitting 12–16 GB consumer GPUs. That makes it a pragmatic approach to get higher reasoning and structured-output quality without moving to much larger, harder-to-run checkpoints.
What Sets It Apart
- Frankenmerge architecture: stacks 32 layers from each source finetune (total 64 layers) to produce an ~18B-parameter model serialized as a Q4_K_M GGUF (~9.2 GB). This lets users run a deeper-capacity model on consumer cards without requiring a full 27B+ checkpoint.
- Heal fine-tune: a 1000-step QLoRA healing pass (NF4, LoRA rank 64) was applied to smooth layer-boundary artifacts; reported improvements include cleaner structured outputs (HTML/CSS/JS), a programming benchmark recovery (11/15 → 12/15), and a substantial loss reduction during tuning.
- Practical benchmarks: authors report 40/44 capability-suite passes and extensive frontend stress tests (62/63 checks passed), highlighting unusually robust long structured outputs for a merged model.
Who It's For (and tradeoffs)
Great fit if you need stronger reasoning or production-quality structured code output but are constrained to 12–16 GB GPUs — e.g., local inference, prototype agent chains, or frontend code-generation pipelines. Look elsewhere if you need fully-supported, production-grade models with exhaustive safety testing: this is an experimental frankenmerge and can still emit formatting glitches (occasional code-block formatting, minor JS/closing-tag artifacts). Also note the GGUF release is the healed Q4_K_M quantized artifact; full BF16 weights are not included in the repo.
Where It Fits
Think of this as a pragmatic middle ground: more capability than a single 9B finetune but far lighter than typical 27–35B checkpoints. It's useful for researchers and practitioners wanting to experiment with merged architectures, heal-fine-tuning effects, or deploy an LLM with improved structured-output behavior on consumer hardware.
