AI Infra2025
Runs and fine-tunes LLMs locally on Apple silicon via the MLX framework, pulling thousands of Hugging Face models with one command. Adds 4- and 8-bit quantization, LoRA and full fine-tuning, prompt caching, and distributed inference across Macs.
AI Infra2025
Transforms unstructured financial content—papers, news, blogs, and filings—into a queryable semantic knowledge graph for retrieval-augmented research. Combines domain-tuned LLMs, embedding-based search, and modular ingestion pipelines; aimed at quant research teams and institutional workflows.
AI Deploy2025
Splits LLM inference into separate prefill and decode GPU pools, then routes requests with KV-cache awareness to cut redundant recomputation. Reports up to 30x throughput on DeepSeek-R1 (GB200 NVL72) and works across TensorRT-LLM, vLLM, and SGLang.
MCP Server2025
Gives an LLM agent direct control of iOS and Android apps over one MCP interface, across simulators, emulators, and real devices. Reads the native accessibility tree to pick elements deterministically, using screenshot coordinates only as fallback.
AI Infra2025
Federates MCP, A2A, and REST/gRPC backends behind a single gateway endpoint with centralized discovery, governance, and observability; optimizes agent and tool calling. Includes gRPC→MCP translation, plugin extensibility, OpenTelemetry tracing, and Kubernetes-ready deployment.
AI Infra2025
Converts PDFs into AI-ready structured outputs (Markdown, JSON with bounding boxes, HTML) for RAG and accessibility workflows; offers deterministic local parsing plus a hybrid AI mode for complex tables, OCR, formulas, and auto-tagging previews.
AI Agent2025
Packages an AI agent's memory — data, embeddings, search indexes, and metadata — into one portable .mv2 file, replacing multi-service RAG stacks. Combines BM25 and HNSW search with temporal queries and sub-millisecond local reads, fully offline.
AI Infra2025
Parses the local JSONL logs that coding-agent CLIs write and turns them into token and cost reports, no API keys or telemetry. Breaks spend down by day, month, session, and Claude's 5-hour billing windows across Claude Code, Codex, Gemini CLI and more.
AI Agent2025
Wraps Claude Code as an MCP server that orchestrates 100+ specialized agents into self-organizing swarms — hierarchical, mesh, or adaptive consensus — backed by persistent vector memory, coordination hooks, and secure cross-machine federation.
AI Infra2025
Provides a community-curated database of AI model metadata—specs, pricing, and capabilities—and exposes it via a JSON API and a TOML-based contributor workflow for programmatic lookup and integration.
AI Infra2025
Extends RAG beyond text: parses PDFs and Office files containing images, tables, equations, and charts, then queries them through one multimodal knowledge graph. Built on LightRAG, it replaces separate parsing and retrieval tools.
AI Infra2025
Reimplements the vLLM inference engine from scratch in ~1,200 lines of readable Python, matching its offline throughput on small models. Prefix caching, tensor parallelism, torch.compile, and CUDA graphs are all kept legible.