MLOps2018
Orchestrates and schedules Python data pipelines and workflows with primitives for retries, caching, parameters, and deployments. Provides either a self-hosted server or managed Prefect Cloud for monitoring, observability, and integrations across common data tools.
AI Deploy2018
Serves machine learning and deep learning models for cloud, data center, edge and embedded environments. Supports multiple frameworks and backends, dynamic and sequence batching, HTTP/gRPC APIs, Docker deployment and NVIDIA-optimized runtimes.
AI Infra2018
Runs ONNX models faster on CPU, GPU, and NPU by routing graph subgraphs to backend execution providers (CUDA, TensorRT, OpenVINO, DirectML, CoreML). One engine serves the same model across cloud, browser, mobile, and edge, for both inference and training.
AI Deploy2019
Turns Python ML code into production inference APIs that scale on Kubernetes or any cloud. Bundles models, dependencies, and serving logic into versioned "Bentos" with autoscaling, scale-to-zero, and multi-GPU serving for LLMs and custom models.
AI Audio2019
Provides a toolkit and codebase for building, training, and deploying speech and multimodal models — Automatic Speech Recognition, Text-to-Speech, and speech-aware LLMs — with modular neural components and pre-trained checkpoints for PyTorch. Supports streaming/low-latency inference, multi-language models, and optional compiled kernels for acceleration.
AI API2020
Manages OAuth, credential storage, API proxying, and deployable TypeScript integration functions so products and AI agents can access 800+ external APIs. Includes AI-assisted function generation, a production runtime with scaling and observability, and cloud or self-hosted deployment options.
AI Infra2020
An AI-native, weight-centric infrastructure for quantitative trading that produces target portfolio weight vectors to unify data ingestion, strategy composition, backtesting, and live/broker execution. Modular pipeline supports ML/DRL allocators, LLM-ready preprocessing, multi-source data, and Alpaca integration for paper/live trading.
AI Deploy2021
Serves predictive and generative ML models on Kubernetes via a single InferenceService CRD, with scale-to-zero, canary rollouts, and an OpenAI-compatible LLM path on vLLM. One autoscaling abstraction over PyTorch, XGBoost, ONNX, and HuggingFace.
MLOps2021
Orchestrates ML training pipelines and production agent workflows from one Python codebase that runs unchanged from a laptop to Kubernetes or any cloud. Auto-versions artifacts, models, and agent checkpoints, with no orchestrator or framework lock-in.
AI Agent2022
Framework for building multi-channel AI assistants that autonomously plan tasks, invoke tools/skills, and keep long-term memory; supports many LLM providers and channels (WeChat, Feishu, QQ, web) for local or server 24/7 deployment.
AI Model2022
Rust-and-Python toolkit that serves open-source LLMs (Llama, Falcon, Mixtral, StarCoder) over HTTP/gRPC with tensor parallelism, continuous batching, Flash/Paged Attention and quantization. Now in maintenance mode, pointing users toward vLLM and SGLang.
AI Infra2022
Build full‑stack web apps entirely in Python — write frontend components and backend state as Python classes with a reactive model. Provides fast refresh, deployment tooling, and AI-focused integrations such as an AI Builder and an Agent Toolkit for connecting LLMs and image models.