AI Audio2024
Lets AI agents place and answer business phone calls, holding spoken conversations to collect structured data, answer questions, and escalate to humans. Built on Azure Communication Services and Azure OpenAI, with RAG over your own documents.
AI Model2024
Performs document OCR, layout analysis, reading-order detection and table recognition across 90+ languages using a ~650M-parameter vision–language model; offers per-page and per-block modes and supports GPU (vllm) and CPU/Apple Silicon backends.
AI Deploy2024
GPU kernel library for LLM inference attention, sampling, and KV-cache, built on block-sparse formats with JIT-compiled customizable templates. Reports 29-69% inter-token-latency cuts vs compiler backends; powers SGLang, vLLM, and MLC-Engine.
AI Infra2024
Runs AI-generated code in isolated, elastic sandboxes with SDK, API, and CLI access for agent workflows that need stateful execution and environment control.
Chatbot2024
Runs a privacy-first, self-hosted answering engine that combines web retrieval with local and cloud LLMs to produce cited answers. Supports SearxNG search, file uploads, image/video search, and mix-and-match models with Speed/Balanced/Quality modes.
AI API2024
Provides local inference, fine-tuning, and a server/CLI for vision–language and omni (image/audio/video) models via MLX. Supports multi-image chat, audio/video inputs, activation quantization (CUDA), TurboQuant KV cache, and LoRA/QLoRA fine-tuning for on-device workflows.
AI Deploy2024
Applies quantization, pruning, distillation, speculative decoding and sparsity to compress and accelerate deep learning models, producing exportable checkpoints ready for deployment on TensorRT‑LLM, TensorRT, vLLM and similar inference runtimes.
AI Client2024
Runs open LLMs entirely on your own machine — discover and download models from Hugging Face, chat in a desktop GUI, or expose an OpenAI-compatible local server. Native Apple MLX and llama.cpp backends; headless deploy via llmster.
AI Image2024
Segments each PDF page into 11 labeled regions — titles, tables, formulas, figures, footnotes and more — and recovers reading order. Offers two engines: an accurate VGT visual model (~0.96 F1) or a faster CPU-only LightGBM ensemble.
AI Infra2024
Stores and reuses LLM key-value caches across GPU, CPU, disk, and remote backends so vLLM and SGLang skip recomputing repeated context. Non-prefix reuse (CacheBlend) and PD disaggregation cut time-to-first-token for long-context and RAG serving.
AI Infra2024
Cloud-native control plane that scales vLLM on Kubernetes, adding the routing, autoscaling, and fault tolerance single-instance serving lacks. Brings high-density LoRA management, an LLM gateway, distributed KV cache reuse, and SLO-aware GPU serving.