Turns clinical text into structured, de-identified clinical signals—entity extraction and PII de-identification—that run entirely on local hardware. Provides 1,000+ specialized medical NER models, multilingual support, Apple MLX acceleration, and Apache‑2.0 licensing.
Provides mined hard negatives and relevance scores for 1.88M queries across seven retrieval datasets, enabling contrastive fine-tuning and nv-retrieve filtering; includes full 2048 mined negatives per query, paired query/document splits, and parquet-formatted files for large-scale training.
Aggregates SEC EDGAR filings into raw files, parsed plaintext, and rich filing metadata for LLM training and retrieval. Includes ~8.05M filings (~590 GB, ~43B tokens), per-filing token counts, and parsed outputs; Apache-2.0.
Generates summaries from URLs, YouTube videos, podcasts, PDFs, and local audio or video files. Backend-agnostic by design: the same pipeline drives local coding CLIs (Claude, Codex, Gemini) or hosted API providers (OpenAI, Google, xAI).
Provides a plug-and-play inference engine that lets language models programmatically inspect, decompose, and recursively call themselves to handle very long contexts; supports local and cloud REPL sandboxes, multiple LLM backends, and trajectory logging/visualization.
Provides open ASR and TTS speech data for 24 Sub‑Saharan African languages to train and evaluate speech models. Includes ~1,250 hours of transcribed ASR and ~235 hours of single‑speaker TTS with train/validation/test/unlabeled splits and mixed CC-BY licenses.
Processed, multilingual news corpus of 1.357B articles extracted from Common Crawl CC‑News with per-article WARC provenance. Includes trafilatura-extracted bodies, language labels (GlotLID & CommonLingua), IPTC topic tags, monthly Parquet shards and a companion FM-index for sub-10ms substring queries; bulk text access is gated for academic research.
Provides L3 refined synthetic training data by converting high-quality web corpora into Q&A pairs and multi-style rewrites; supplies 400B+ English and 200B+ Chinese tokens for late-stage LLM pretraining and decay-phase training.
Provides multi-task long-speech evaluation data for eight speech-understanding tasks (ASR, summarization, QA, translation, emotion, speaker counting, content separation, language detection). Includes 101,822 long audio files and ~204,881 annotated examples with JSONL task splits for easy loading.
Filtered subset of the OPUS 4.6 parallel corpus that isolates reasoning-related translation examples and removes 979 refusals, providing a cleaner 3,000×-filtered dataset for training or evaluating NLP models focused on reasoning in translation.
Cleaned reasoning dataset of problem→thinking→solution triplets derived from Opus 4.6, provided in Parquet with ~2,160 cleaned rows (original 3,305). Filters remove empty/short/refusal/non‑substantive responses; hosted on Hugging Face under Apache‑2.0.
JSONL dataset of Claude Opus 4.6 chain-of-thought traces paired with high-difficulty math and logic problems for supervised fine-tuning and distillation; exposes step-by-step reasoning to teach process-oriented problem solving and improve math/logic accuracy in smaller LLMs.