MLC-LLM

Universal LLM deployment engine that compiles models with TVM Unity for native execution across GPUs, CPUs, mobile and WebGPU.

Visit Website

Introduction

Overview

MLC-LLM turns any HuggingFace checkpoint into a highly-optimized library with int4/int8 kernels, delivering OpenAI-style APIs on desktop, mobile and browser.

Key Capabilities

Ahead-of-time compilation via TVM Unity
Unified engine with Metal, Vulkan, CUDA back-ends
REST, Python, JS, iOS & Android SDKs
WebLLM for client-side web inference

Back

Information

Websitellm.mlc.ai
AuthorsMLC AI Lab
Published date2023/04/29

More Items

Ray

2017

RISELab (UC Berkeley), Anyscale Inc.

Ray is an open-source distributed compute engine that lets you scale Python and AI workloads—from data processing to model training and serving—without deep distributed-systems expertise.

ai-development ai-framework ai-train ai-serving

OpenVINO

2018

Intel

OpenVINO is an open-source toolkit from Intel that streamlines the optimization and deployment of AI inference models across a wide range of Intel® hardware.

ai-development ai-inference ai-serving

NVIDIA Dynamo

2025

NVIDIA

NVIDIA Dynamo is an open-source, high-throughput, low-latency inference framework that scales generative-AI and reasoning models across large, multi-node GPU clusters.

ai-development ai-inference ai-serving nvidia