AI Audio2023
Terminal CLI for on-device Whisper ASR using Hugging Face Transformers + Optimum, with optional Flash Attention 2, batching, and diarization support — focused on high-throughput transcription on NVIDIA GPUs and Apple Silicon (mps).
AI Deploy2023
Compiles LLMs into optimized TensorRT inference engines for NVIDIA GPUs via a Python API. Layers in kernel fusion, quantization, paged attention, KV caching, and continuous in-flight batching, scaling from a single GPU to multi-node deployments.
AI Client2023
Hands-free voice-first companion with a Live2D avatar for real-time conversations with LLMs. Cross-platform web and desktop clients, runs locally or via cloud APIs, supports local ASR/TTS and modular customization for personas and models.
AI Infra2023
Provides a NumPy-like array framework for building and training ML on Apple silicon, with Python, C/C++, and Swift APIs plus PyTorch-style higher-level modules. Features lazy evaluation, composable AD/vectorization, and a unified-memory multi-device model so arrays can be used on CPU and GPU without explicit copies.
AI Coding2023
Bundles AI features and coding agents into JetBrains IDEs, using IDE code intelligence for completion, refactoring, and chat. Runs on the proprietary Mellum model or your choice of OpenAI, Gemini, Anthropic, and local models via Ollama or LM Studio.
AI Image2023
Provides a diffusion-model studio for image, video, audio-video, editing, LoRA, and full training workflows so many model families share one inference and training framework.
AI Deploy2023
Builds custom AI inference servers in pure Python on top of FastAPI, keeping full control over request logic while batching, GPU autoscaling, streaming, and OpenAI-spec endpoints come built in. Claims a 2x+ throughput edge over plain FastAPI.
AI Infra2023
Triton kernels and PyTorch layers for linear-attention, state-space, and sparse-attention token mixers (GLA, RWKV, Mamba2, GSA) as drop-in replacements for multihead attention. Runs on NVIDIA, AMD, and Intel GPUs with Hugging Face support.
AI Infra2024
Serves large language and multimodal models with low latency and high throughput using RadixAttention, continuous batching, structured outputs, parallelism, quantization, and broad accelerator support.
AI Model2024
Performs document OCR, layout analysis, reading-order detection and table recognition across 90+ languages using a ~650M-parameter vision–language model; offers per-page and per-block modes and supports GPU (vllm) and CPU/Apple Silicon backends.
AI Audio2024
Clones a voice from a 5-second sample for zero-shot TTS, or fine-tunes on ~1 minute of audio for few-shot synthesis. Covers Chinese, English, Japanese, Korean, and Cantonese, with a WebUI bundling vocal separation, ASR, and dataset labeling.
AI Image2024
Reworks AUTOMATIC1111's Stable Diffusion WebUI onto a custom backend that auto-manages GPU memory to speed inference and cut VRAM use. Adds native FLUX support with NF4/GGUF quantization and a UNetPatcher framework for model-agnostic extensions.