AI Infra2024
Converts PDFs, images, and Office documents into Markdown or JSON for retrieval, extraction, and agent workflows, with OCR, layout analysis, formula handling, and multiple runtime modes.
AI Agent2024
Local-first runtime for autonomous AI agents that run on-device and stay model-agnostic across OpenAI, Anthropic, Gemini, Grok, and local models. A plugin system adds chat platforms (Discord, Telegram, X), voice, browser automation, RAG, and wallets.
AI Infra2024
Drives UI automation from screenshots alone: describe steps in natural language and a vision model acts on what it sees, no DOM selectors. One API spans web, Android, iOS, HarmonyOS and desktop; plugs into Playwright/Vitest or runs autonomously.
AI Agent2024
Open-source platform for autonomous coding agents that work like developers: editing files, running shell commands, browsing the web, and calling APIs in an isolated sandbox. Model-agnostic, with GitHub, Slack, and CI/CD integration.
AI Infra2024
kvcache-ai (maintainers / MADSys group, Tsinghua University & contributors)MADSys Lab, Tsinghua University, Approaching.AI +1
Runs huge mixture-of-experts LLMs like DeepSeek-R1/V3 on a single 24GB GPU plus CPU DRAM by keeping attention on the GPU and offloading expert weights to CPU. Reports 3-28x speedups via Intel AMX/AVX512 kernels and fits 139K context in 24GB VRAM.
AI Infra2024
Official inference framework for 1-bit and ternary (1.58-bit) LLMs such as BitNet b1.58, with optimized CPU kernels. Delivers 1.37x-6.17x speedups and 55-82% lower energy on x86 and ARM, and runs a 100B model on a single CPU at 5-7 tokens/sec.
AI Infra2024
Runs local LLM, vision-language, ASR, OCR, and image-generation models across NPU, GPU, and CPU from one command. Differs from Ollama and llama.cpp with first-class Qualcomm Hexagon NPU support and day-0 coverage of new models like Qwen3-VL.
AI Infra2024
Converts PDFs, Office files, HTML, images and audio into one structured DoclingDocument, with deep PDF layout, reading order, table-structure and formula recognition, OCR, and native LangChain/LlamaIndex/Haystack integrations for RAG pipelines.
AI Client2024
Runs AI models on user devices with native SDKs, optimized model management, hardware acceleration, and OpenAI-compatible APIs for apps that need offline, private inference.
AI Infra2024
Lets Python developers write tile-based parallel kernels for NVIDIA GPUs, generating CUDA Tile IR while staying close to Python syntax for custom GPU operations.
AI Infra2024
A compact domain-specific language for writing high-performance GPU/CPU kernels (GEMM, FlashAttention, sparse kernels) with Python-like syntax. It provides tiling/pipelining primitives, a TVM-based compiler and multiple backends (CUDA/CuTeDSL, NVRTC, WebGPU, Metal, Ascend) for operator-level performance work.
AI Infra2024
Runs untrusted workloads inside hardware-isolated local microVMs with Docker-like workflows; cross-platform and OCI-compatible, embeddable via SDKs and optimized for fast startups (typical guest boot <100 ms).