AI Dataset2024
Provides a multilingual, deduplicated corpus of public source code in Parquet for large-scale model training and evaluation. Includes license metadata, language splits, and streaming-friendly packaging for use with Hugging Face Datasets — suited to training code-focused foundation models but requires careful license/provenance review.
AI Dataset2024
Provides 300k annotated multilingual text examples for identifying and masking personally identifiable information (PII) across multiple domains and languages (EN, FR, DE, IT, ES, NL). Intended for training and evaluating token-level PII detection and masking models; includes a DOI for citation.
AI Dataset2024
Hisami Suzuki, Satoru Katsumata +4LLM-jp (Center for Large Language Model Research and Development, National Institute of Informatics), National Institute of Informatics
Provides manually curated Japanese instruction pairs (questions and safe reference answers) for improving LLM output safety, covering broad harm categories and regionally sensitive cases. Includes English meta-tags and standard splits for benchmarking and fine-tuning.
AI Model2024
An instruction-tuned 8B Llama 3.1 model for multilingual conversational text generation, built for assistant-style chat and long-context inputs (up to 128k tokens). Available for use via the Transformers pipeline and inference endpoints, with common optimizations like safetensors.
AI Audio2024
Local WebUI for video and audio dubbing: download YouTube, isolate vocals, transcribe with Whisper, translate into 100+ languages, and generate multilingual TTS or zero-shot voice clones. Uses Gradio, yt-dlp, Demucs, Whisper/Faster-Whisper, F5/E2/CosyVoice and Edge-TTS; Windows-focused with optional Azure integration.
AI Dataset2024
Provides pre-parsed Parquet snapshots of English and French Wikipedia articles with structured fields (sections, infoboxes, tables, references, images) and credibility signals — optimized for large-scale analysis, retrieval-augmented generation, and model development.
AI Audio2024
Provides low‑latency on‑device speech-to-text, intent recognition, and text-to-speech for building real‑time voice agents and interfaces. Streaming-optimized models, incremental caching, multilingual TTS/ASR and cross-platform bindings (Python, iOS, Android, Linux, Raspberry Pi) target live voice use cases where sub-200ms responsiveness matters.
AI Audio2025
Zero-shot, single‑reference voice cloning TTS with multilingual support (ZH/EN/JA/ES/AR), fine-grained emotion and duration control, and pronunciation hooks (Pinyin/CMU/Kana); ships model weights, Web UI and production deployment recipes for local or server use.
AI Audio2025
Provides real-time, local audio recording and transcription on macOS using Whisper and Parakeet engines, with global hotkeys and hold-to-record behavior. Includes model download, microphone selection, drag-and-drop file transcription, multilingual auto-detection and Asian-language autocorrect; Apple Silicon only.
AI Dataset2025
Provides professionally translated parallel corpora and a multilingual lexicon across 100+ low-resource languages for training and evaluating multilingual MT and NLP models. Includes SmolDoc, SmolSent, GATITOS, and factuality annotations; licensed CC-BY-4.0.
AI Dataset2025
High-quality, efficiently verified and filtered web corpus for LLM pretraining — supplies ~1 trillion English tokens and ~120 billion Chinese tokens with English/Chinese Parquet splits. Designed for large-scale pretraining experiments and data-filtering research.
AI Client2025
Turns web reading into an in-context language-learning experience by injecting context-aware translations, explanations, subtitle translation, and TTS directly into the browser. Supports selection translation, batch requests and configurable AI providers to balance cost and quality.