Fixing recurrent looping in reasoning LLMs matters because repetition undermines usability for long-form thinking and deterministic decoding. This v2 release targets that specific hygiene problem: it keeps the deep chain-of-thought behavior and long-context capability of the original Qwythos while removing the degeneration that caused repeated paragraphs under greedy or low-temperature decoding.
Key Capabilities
- Looping elimination with minimal impact: Final-Token Preference Optimization (FTPO) trains the model to prefer coherent alternatives at loop-start tokens, dropping greedy looping from 6.7% to 0.0% while preserving reasoning accuracy. This is a targeted, light-touch intervention rather than broad overfitting.
- Preserved reasoning and benchmarks: internal CoT-style evaluations report strong performance (e.g., MMLU 83.8% with CoT, GSM8K 93.6%, HumanEval ~77%). The release is positioned as a robustness/hygiene upgrade, not a capability leap.
- Long-context and multimodal stack: YaRN scaling provides a 1,048,576-token context window out of the box (static factor 4), and the architecture retains the hybrid Qwen3.5-9B attention design with a restored native MTP head for speculative decoding workflows.
- Practical serving notes: recommended sampling defaults favor reasoning (temperature ~0.6, top_p 0.95, top_k 20) and max_new_tokens sized for extended analyses; safetensors bfloat16 weights and common Transformers/vLLM compatibility make deployment straightforward.
Who it's for and trade-offs
Great fit if you need an on-prem / research-facing LLM that: reliably produces long-form chain-of-thought without greedy degeneration; accepts very long contexts; and permits targeted red-teaming or sensitive technical research. Look elsewhere if you require an extensively safety-filtered model by default, the absolute top HumanEval score (v2 trades a couple points vs some bases), or if you cannot accept an intentionally uncensored research posture. The FTPO fine-tune is intentionally small (≈2k preference tuples, light LoRA-style hyperparams) so some micro-differences vs base-model behavior can exist.
How the loop fix works (brief)
Rather than globally reshaping output distributions, FTPO identifies the exact token position that initiates a recurrent loop and nudges the model to prefer coherent alternatives at that position. The rest of the model distribution and knowledge is left largely intact, which minimizes capability regression while removing common failure modes under deterministic decoding.
