AI Train2023
Collection of runnable model implementations — LLaMA, Mistral, Stable Diffusion, Whisper, CLIP, plus LoRA fine-tuning — ported to the MLX array framework so they run natively on Apple silicon's unified memory rather than CUDA.
AI Infra2023
Traces how Transformer LLMs route information from input to output, attributing each block's effect to individual attention heads and feed-forward neurons. Click any edge to see what a head promotes or suppresses in vocabulary space.
AI Dataset2023
A multiple-choice benchmark for evaluating LLM understanding in Traditional Chinese across 66 subjects (elementary to professional). Contains ~22K verified questions covering STEM, humanities, social sciences and Taiwan-specific topics, with standardized splits and model leaderboards under an MIT license.
Re-derives LLM scaling laws, tracing prior disagreements to how compute budget was modeled, then trains 7B and 67B models on 2T tokens. The 67B model beats LLaMA-2 70B on code, math, and reasoning; its chat variant tops GPT-3.5 on open-ended evals.
Reworks the MoE layer to push each expert toward a narrow specialty: split experts into many finer ones and activate more per token, plus reserve a few always-on shared experts for common knowledge. A 2B model matches GShard 2.9B; at 16B it rivals LLaMA2 7B on ~40% of the compute.
A family of open code models (1.3B-33B) trained from scratch on 2T tokens of project-level code, using a 16K-window fill-in-the-blank objective. Beats Codex and GPT-3.5 on code benchmarks and ships under a license permitting commercial use.
AI Train2024
Runs GPT-4o-class vision, speech, and full-duplex audio-video conversation on a 9B model small enough to deploy on phones and tablets. The 4.5 release scores 77.6 on OpenCompass and adds real-time bilingual voice with voice cloning.
Reaches 51.7% on the competition-level MATH benchmark with a 7B model and no tools or voting, rivaling Gemini-Ultra and GPT-4. Built on a 120B-token math corpus mined from Common Crawl, and introduces GRPO, a memory-efficient PPO variant for reasoning.
AI Dataset2024
Provides a multilingual, deduplicated corpus of public source code in Parquet for large-scale model training and evaluation. Includes license metadata, language splits, and streaming-friendly packaging for use with Hugging Face Datasets — suited to training code-focused foundation models but requires careful license/provenance review.
AI Agent2024
Generates Wikipedia-style articles from web search using LLMs: it researches topics, produces multi-perspective outlines via perspective-guided question asking and simulated conversations, then drafts full articles with citations. Supports human-AI collaboration (Co-STORM) and grounding with multiple retrievers.
AI Dataset2024
Provides 300k annotated multilingual text examples for identifying and masking personally identifiable information (PII) across multiple domains and languages (EN, FR, DE, IT, ES, NL). Intended for training and evaluating token-level PII detection and masking models; includes a DOI for citation.