Introduces Draft-OPD, an on-policy distillation method for training lightweight draft models used in speculative decoding — it focuses learning on draft-induced errors via target-assisted rollouts and replay, improving acceptance length and enabling >5× lossless LLM inference acceleration.
Analyzes when masking stale observations improves long-horizon search agents and why, identifying an asymmetric inverted-U relationship between masking benefit, retriever quality, and model capacity; explains a token-for-turn trade-off and releases evaluation scaffolds and trajectories.
A GGUF-quantized, locally runnable build of Gemma 4 12B Unified (image-text-to-text) packaged by unsloth; preserves multimodal (image/audio) input support under an Apache-2.0 license and is compatible with common GGUF runtimes and Unsloth Studio.
Automates distillation of heterogeneous traces from a target person or role into versioned, inspectable skill packages for LLM agents — producing separate capability and bounded-behavior tracks that support natural-language corrections, rollback, and cross-host installation. Ships with an open system and a skills gallery.
Uses search-agent reading traces and tiered distractors to train LLMs for long-context, multi-hop reasoning, and introduces a rubric reward that supervises entity-level steps (applied only to correct finals). Improves evidence-grounded reasoning and resists reward hacking across 4B–30B models.
Provides ~3.8M chain-of-thought training examples (~3B tokens) that use a fixed 'Inventory / Interaction / Execution' scaffold to improve concise, reliable reasoning in small LLMs; topic-labeled, deduped, includes real tool-execution traces, CC-BY-4.0.
A 16 GB, 507-file PhD‑level cybersecurity knowledge base for training and evaluating security-focused LLMs and automation. Covers offensive/defensive/forensics/cloud/iot and AI-security across 30+ domains with real-world labs and framework mappings.
Provides 462 unrestricted long-form chain-of-thought reasoning traces distilled from the full Mythos V2 model (≈104.7M characters); intended for long-context evaluation, trace analysis and process-level supervision. License unknown—verify before reuse.
Proposes TrOPD, a method that restricts token-level on-policy distillation to regions where teacher supervision is reliable to stabilize training under teacher–student distribution mismatch. Adds outlier handling (clipping, masking, forward-KL) and off-policy guidance; shows consistent gains on math reasoning, code generation and general benchmarks.
Studies small trainable adapters (PEFT) used as persistent personal models on top of large foundation models, analyzing three scaling axes—Scale Up, Scale Down, Scale Out—and introducing MinT, an infrastructure for adapter identity, provenance, evaluation, and serving.
Localizes harmful span-level errors inside long research-agent trajectories to show which trajectory segments make final answers unreliable. Provides a 1,000-instance TELBench of annotated spans and DRIFT, a claim-centric auditing method that improves span-level localization and first-error accuracy by up to 30 percentage points.
Analyzes how single-domain RL fine-tuning on LLMs induces cross-domain interference and shows this damage concentrates in a low-dimensional shared conflict subspace; proposes a local perturbation theory and short domain "refresh" procedures that selectively recover earlier domains with minimal collateral loss.