Category

Dream Server

Runs a local-first, full AI stack—LLM inference, chat UI, voice, agents, workflows, RAG, and image generation—deployable with one command. Auto-detects hardware and bootstraps a small model for instant chat while larger models download; supports Linux, Windows, macOS and optional cloud/hybrid modes.

ODS (Osmantic Deployment System)

Turns a PC, Mac, or Linux machine into a private AI server with one-command installers: local LLM inference, a ChatGPT-style web UI, voice, agents, RAG, workflows, image generation, hardware-aware model selection, and optional cloud/hybrid modes.

oMLX

Local LLM inference server for Apple Silicon that exposes an OpenAI-compatible API and a macOS menubar app. Uses continuous batching and a two-tier KV cache (RAM + SSD in safetensors) to persist context across restarts, enabling practical multi-model serving and fast local coding workflows.

Rapid-MLX

Runs local AI models on Apple Silicon as an OpenAI‑compatible server, emphasizing low latency, prompt caching, and reliable tool-calling. Optimized for M1–M4 Macs with multimodal support and drop‑in compatibility for IDEs and agent frameworks.

Openship

Deploy and manage applications and containers to your own servers or Openship Cloud from a single desktop, web, or CLI interface. Built-in CI/CD with push-to-deploy and preview environments, automatic SSL, managed databases, CDN, backups, and multi-node portability for VPS-to-production workflows.

club-3090

Provides reusable Docker compose files, scripts and benchmarked configs to serve modern LLMs (Qwen, Gemma, etc.) on 1–2 NVIDIA RTX 3090/4090/5090 GPUs. Multi-engine (vLLM, llama.cpp, ik_llama), measured TPS/context tradeoffs, and validated single/dual‑GPU recipes.

DeskcommCRM

Self-hosted sales CRM that runs native AI agents (RAG per tenant) to handle WhatsApp conversations, qualify leads, trigger automations and move deals through configurable pipelines. Multi-tenant with LGPD-minded controls and a one-command VPS installer for full data ownership.

Instatic

Self-hosted visual CMS that runs as a single Bun server, combining a canvas editor, content engine, media, auth, forms, plugins, and a publisher. Emits semantic HTML and compact CSS and includes a provider-agnostic AI agent that edits pages as real, editable nodes. Best for teams that want full control and simple deployments.

MTPLX

Runs local LLMs on Apple Silicon using native MTP speculative decoding to accelerate token generation while preserving the model's output distribution. Leverages the model's own MTP heads with batched verification and exact rejection sampling; ships with a Mac app, CLI, local OpenAI/Anthropic-compatible server, auto-tune, and Forge for building/verifying MTP adapters.

Cosmos-Framework

End-to-end Python framework for training and serving NVIDIA's Cosmos world models (Cosmos3), integrating distributed training (FSDP/TP/CP/PP), DCP/safetensors checkpoints, dataset adapters, multiple inference backends, online serving, and agent skills.

TurboServe: Serving Streaming Video Generation Efficiently and Economically

Serves interactive, long-lived streaming video-generation sessions by jointly scheduling session placement and GPU autoscaling to meet tight per-chunk latency. Combines migration-aware placement, load-driven autoscaling, coalesced chunk processing, GPU–CPU offloading and NCCL GPU–GPU migration; reports ~37% reductions in worst-case per-chunk latency and GPU operating cost.

Ornith-1.0-397B

Provides an open-source Mixture-of-Experts coding LLM (397B) optimized for agentic, tool-enabled coding workflows with a 262,144-token context window, OpenAI-compatible API, serving recipes (vLLM/SGLang), and published coding-benchmark results.