AI Dataset2022
Community-curated collection of ChatGPT-style prompts mirrored as a Hugging Face dataset; organized by task and model compatibility for quick reuse. Useful for prompt engineering, text-generation prototyping, and building conversational examples across multiple LLMs.
AI Dataset2023
Contains tech-blog posts scraped from Habr (primarily Russian, some English) in Parquet format with ~100K–1M records. Suited for multilingual text-generation and language-model fine-tuning; license is not specified, so verify before redistribution.
AI Dataset2023
Provides 52,000 English instruction–response pairs generated by OpenAI's text-davinci-003 for instruction-tuning language models. Released under CC BY-NC 4.0; low-cost synthetic data useful for research but contains model-generated biases and errors.
AI Dataset2023
A 15,000+ English instruction–response corpus for fine-tuning and evaluating LLM instruction-following behavior. Contains human-authored prompts and answers across categories (closed/open QA, summarization, extraction, classification, brainstorming) and is released under CC BY-SA 3.0.
AI Dataset2023
Contains short, small-vocabulary stories synthetically generated by GPT-3.5 and GPT-4 for training and evaluating compact language models. Includes multiple splits, a GPT-4-only V2 subset, and archive files with prompts and metadata for reproducible experiments.
AI Dataset2023
A public dataset of one million real-world conversations with 25 LLMs, including conversation text, model name, detected language tags, and OpenAI moderation outputs — useful for studying prompt distributions, safety/moderation, and training/evaluating instruction-following models.
AI Dataset2023
Benchmark for evaluating general AI assistants with 466 short, real-world questions that require tool use, multimodality and reasoning; provides a public dev set and a withheld test set used for leaderboard evaluation.
AI Dataset2023
Provides a 200k-example filtered conversational dataset derived from UltraChat for supervised fine‑tuning and generation‑ranking of chat models. Stored in parquet with four splits and used as part of Zephyr‑7B‑β training data.
AI Dataset2023
A multiple-choice benchmark for evaluating LLM understanding in Traditional Chinese across 66 subjects (elementary to professional). Contains ~22K verified questions covering STEM, humanities, social sciences and Taiwan-specific topics, with standardized splits and model leaderboards under an MIT license.
AI Dataset2024
Provides a multilingual, deduplicated corpus of public source code in Parquet for large-scale model training and evaluation. Includes license metadata, language splits, and streaming-friendly packaging for use with Hugging Face Datasets — suited to training code-focused foundation models but requires careful license/provenance review.
AI Dataset2024
Provides 300k annotated multilingual text examples for identifying and masking personally identifiable information (PII) across multiple domains and languages (EN, FR, DE, IT, ES, NL). Intended for training and evaluating token-level PII detection and masking models; includes a DOI for citation.
AI Dataset2024
Provides a cleaned, deduplicated English web corpus optimized for LLM pretraining—over 15T tokens aggregated from CommonCrawl with per-dump snapshots and smaller sampled configs (10B/100B/350B). Includes the datatrove processing pipeline, MinHash deduplication, and an ODC-By v1.0 license; suited for large-scale model training and ablation studies but not specialized for code.