Introduced the two-stage recipe behind the GPT lineage: unsupervised generative pre-training on unlabeled text, then supervised fine-tuning per task. A single 12-layer Transformer decoder beat bespoke architectures on 9 of 12 NLP benchmarks.
AI Deploy2018
Converts, quantizes, and runs deep learning models from PyTorch, TensorFlow, ONNX, and PaddlePaddle across Intel CPUs, GPUs, and NPUs without the training framework. Adds a GenAI pipeline for LLMs plus Hugging Face, vLLM, and LangChain integrations.
AI Model2018
Provides unified model definitions and a single API for pretrained text, vision, audio, and multimodal models for both training and inference. Emphasizes cross-framework compatibility (PyTorch/TF/JAX), pipeline-based inference, and direct access to 1M+ Hub checkpoints.
AI Model2018
Turns model definitions into a shared layer across training and inference stacks, covering text, vision, audio, video, and multimodal models. Pipelines, Trainer, and generation APIs make pretrained models usable without locking teams to one framework.
AI Deploy2019
Turns Python ML code into production inference APIs that scale on Kubernetes or any cloud. Bundles models, dependencies, and serving logic into versioned "Bentos" with autoscaling, scale-to-zero, and multi-GPU serving for LLMs and custom models.
A 1.5B-parameter model trained only to predict the next token on diverse web text does translation, summarization, and QA zero-shot, with no fine-tuning. It recast NLP tasks as conditional language modeling and sparked the staged-release misuse debate.
AI Agent2019
Open-source Airtable alternative for building databases, apps, automations, and AI agents without code over a PostgreSQL-backed REST API. The Kuma assistant turns plain language into tables and workflows; self-hostable with full data ownership.
Demonstrated that language model loss falls as a smooth power law in model size, data, and compute across more than seven orders of magnitude — turning "make it bigger" from a hunch into a budget you can plan, and justifying the GPT-3 scale-up.
At 175 billion parameters, this autoregressive model becomes a strong few-shot learner: it handles translation, QA, and reasoning from a few prompt examples with no gradient updates, establishing in-context learning as an alternative to fine-tuning.
AI Infra2020
An AI-native, weight-centric infrastructure for quantitative trading that produces target portfolio weight vectors to unify data ingestion, strategy composition, backtesting, and live/broker execution. Modular pipeline supports ML/DRL allocators, LLM-ready preprocessing, multi-source data, and Alpaca integration for paper/live trading.
AI Infra2020
Unified framework for few-shot evaluation of generative language models across 60+ academic benchmarks. Supports multiple model backends (Hugging Face, vLLM, APIs, local servers), configurable prompts/YAML configs, and reproducible exports for leaderboards and research comparisons.