Presents an open 7B dense foundation model for mathematical reasoning and tool-assisted web search, supporting 256K-token context. Key features: hybrid gated sliding-window + global attention for efficient long-context training, FP8 Muon optimizer, progressive 16K→64K→256K curriculum, and open-source weights, data, and recipes.
Reframes Policy Mirror Descent for autoregressive LLM generation into a critic-free, trajectory-level objective using the Bellman equations to avoid per-token value estimation. Derives a practical token-level loss with a smoothed mismatch-correction weight and shows improved math-reasoning accuracy on benchmarks.
Analyzes why on-policy distillation can produce excessively long student rollouts by identifying termination-token mismatch between students and teachers, and proposes aggregating functionally equivalent EOS tokens as a shared semantic stopping action to mitigate length inflation.
Learns dynamics-aligned latent representations for parametric PDE forecasting using masked-latent predictive pretraining, a geometry projector that aligns latent trajectory geometry, and a physics-structured latent predictor that separates shared evolution from parameter-dependent responses.
Analyzes why self-evolving reasoning models collapse under repeated self-training and proposes R-Quest: a feedback-driven pipeline that trains solvers to reject invalid questions and uses a frozen base model to detect task-level repetition, filtering training data to sustain multi-round gains.
Provides 369 Harbor sandbox tasks ported from OpenAI's openai/math: each task is a Lean theorem with missing `sorry` proofs that an agent must complete, graded by a strict Comparator exact-match verifier. Includes task definitions, generator, and manifest for RL/code-agent evaluation.
Improves token-level credit assignment for RL fine-tuning of language models by parameterizing the critic as cumulative, position-wise log-odds increments (Tracking-Value PPO), enabling more accurate value-evolution tracking under sparse terminal rewards and boosting math-reasoning and coding benchmarks.
Restores recoverability in on-policy distillation by having the teacher generate critiques of incorrect student rollouts, guide a critique-free student rewrite, and score the revised solution as privileged context; yields improved math-reasoning performance, largest gains on competition problems.
Compiles a deterministic RISC-V CPU into a language model’s weights, enabling exact arithmetic and C or MicroPython execution without an external tool runtime. It uses Gated DeltaNet state as memory and remains experimental.
Estimates how committed tokens improve predictions at remaining masked positions, then uses those contributions to refine reinforcement-learning signals for diffusion language models. On LLaDA-8B-Instruct, it raises MATH500 and GSM8K scores by up to 9.0 and 5.3 points.
Turns university mathematics textbooks into structured proof-training data while preserving chapter order, proposition–proof links, and dependencies. The authors report a 16.7-point average gain over the base model across six evaluations.