Tag

Apache Airflow

Programmatically author, schedule, and monitor data workflows as Python-defined DAGs; the scheduler handles dependencies, retries, and backfills. Pluggable executors (Local, Celery, Kubernetes) and a broad provider ecosystem for AWS, GCP, and databases.

Crawlee

Unified Node.js library for web crawling and browser automation that fetches pages and files via headless browsers or raw HTTP. Provides persistent queues, proxy rotation, session management, storage, and human-like fingerprints to build scalable data pipelines (e.g., RAG/LLM datasets).

TensorRT

Converts trained PyTorch, TensorFlow, and ONNX models into GPU-tuned inference engines via layer fusion, kernel auto-tuning, and reduced precision. Cuts latency, raises throughput on NVIDIA GPUs from Turing (INT8), with FP8 on Ada+ and FP4 on Blackwell+.

openpilot

Upgrades supported cars with an open robotics operating system for driver assistance, combining vehicle integration, safety testing, data collection, and community development around comma.ai hardware.

roadmap.sh (Developer Roadmap)

Interactive, community-driven learning roadmaps and guides that map skills, technologies, and curated resources for developer career paths — covering frontend, backend, DevOps, ML/AI, MLOps, prompt engineering and more. Clickable nodes link to tutorials, best practices and question banks to guide study and hiring prep.

Nx

Manages polyglot monorepos by caching unchanged outputs and running only affected tasks. Built with Rust and extensible in TypeScript; includes integrated CI features (remote caching, task distribution) and AI-native tooling such as a CLI optimized for autonomous agents and self-healing CI.

Ray (by Anyscale)

Scales any Python or ML workload across CPUs and GPUs with a few decorators, instead of rewriting code for Spark or MPI. Bundles libraries for distributed training, hyperparameter tuning, RL, batch inference, and online model serving on one cluster.

Streamlit

Turns a top-to-bottom Python script into an interactive web app: each widget interaction reruns the whole script, with cache decorators skipping redundant work. No callbacks or HTML needed; built for data dashboards, ML demos, and internal tools.

MLflow

Tracks ML and LLM experiments end to end: logs params, metrics, and artifacts, versions models in a registry, and records agent traces via OpenTelemetry. Framework-agnostic, runs locally or self-hosted, with 50+ built-in evaluation metrics and LLM judges.

NautilusTrader

Rust-native, event-driven trading platform for backtesting and live execution across crypto, forex, equities, and futures on 27+ venues. The same strategy code runs in nanosecond backtests and in production, giving true research-to-live parity.

JAX: High performance array computing — JAX documentation

Turns NumPy-style Python into differentiable, compiled, vectorized programs for CPU, GPU, and TPU. Its leverage is composable transformations: grad, jit, vmap, and sharding combine instead of living in separate APIs.

MLRun

Turns plain Python functions into versioned, serverless ML jobs that run unchanged locally or on Kubernetes, with built-in tracking and deployment. Its feature store derives both offline (batch) and online (real-time) serving from one definition.