Ingests documents, images, audio, video and web pages and converts them into structured, LLM-friendly markdown and parsed data. Runs locally (fits on a T4 GPU), supports ~20 file types, offers OCR, transcription, table extraction and a Gradio UI; deployable via Docker/Skypilot. Licensed under GPL-3.0; some model weights carry cc-by-nc-sa restrictions for commercial use.
Official code companion to the O'Reilly book by Jay Alammar and Maarten Grootendorst: 12 chapters of runnable notebooks on tokens, embeddings, Transformers, text classification, clustering, prompt engineering, semantic search, RAG, and fine-tuning.
Multi-tenant agent harness that makes enterprise knowledge retrievable, graph-reasonable, and deliverable by LLM-powered agents. Integrates RAG + a Milvus-based knowledge graph, LangGraph orchestration, and document parsing for citation-backed answers and graph reasoning; deployable via Docker (requires a compatible LLM API).
Local-first runtime for autonomous AI agents that run on-device and stay model-agnostic across OpenAI, Anthropic, Gemini, Grok, and local models. A plugin system adds chat platforms (Discord, Telegram, X), voice, browser automation, RAG, and wallets.
Connects any LLM to internal knowledge sources and lets teams chat with cited, RAG-style answers. Notable for broad connectors (Drive, Notion, GitHub, YouTube), universal LLM/embedding support, and self-hostable Docker deployment — aimed at teams that need private, searchable LLM-backed knowledge.
Developer framework for building AI agents that autonomously trade on Polymarket prediction markets. Bundles the Polymarket and Gamma APIs, a Chroma RAG layer that pulls in news, and a CLI to query markets, reason with an LLM, and execute trades.
Stores agent memory as human- and agent-readable Markdown files with wikilinks instead of an opaque vector DB. Auto Memory/Resource/Dream jobs distill conversations into long-term notes, and hybrid wikilink + BM25 + embedding search retrieves them.
Provides pre-parsed Parquet snapshots of English and French Wikipedia articles with structured fields (sections, infoboxes, tables, references, images) and credibility signals — optimized for large-scale analysis, retrieval-augmented generation, and model development.
Converts PDFs, Office files, HTML, images and audio into one structured DoclingDocument, with deep PDF layout, reading order, table-structure and formula recognition, OCR, and native LangChain/LlamaIndex/Haystack integrations for RAG pipelines.
Parses, generates, and filters training data from noisy sources like PDFs and weak QA, then feeds it into LLM pre-training, SFT, RL, or RAG cleaning. Ships 100+ operators and ready-made pipelines for text, reasoning, Text2SQL, and agentic data.
Self-hostable alternative to Google NotebookLM: organize PDFs, videos, audio, web pages, and Office docs, then chat over them, take AI-assisted notes, and search via full-text and vector. Routes to 18+ model providers and generates 1-4 speaker podcasts.
Framework for building and orchestrating multi-agent LLM systems, with agent types, tool integration, and human-in-the-loop workflows. Supports multi-agent conversation patterns, multiple LLM providers, and RAG-style tooling for research and prototyping agentic workflows.