AIAny
AI Infra2023
Icon for item

The Data Engineering Handbook

Curated learning hub that aggregates roadmaps, tutorials, bootcamps, books, projects, and tool recommendations for learning data engineering and production data infrastructure. Focuses on practical applied learning (projects, interview prep, community links) rather than code libraries.

Introduction

Most people learning data engineering struggle to find a single, up-to-date place that connects fundamentals (ETL, storage, warehouses), modern lakehouse patterns, and the operational practices that production ML/AI teams need. This handbook consolidates those threads into a navigable learning path and resource index prioritized for applied skills and job readiness.

What Sets It Apart
  • Comprehensive curated index: organizes roadmaps, free bootcamps, project ideas, interview prep, books, newsletters, and community links so learners can move from fundamentals to production topics without hunting across dozens of disparate blogs and repos.
  • Practical focus on infrastructure that matters to ML/AI: includes recommended tools and vendors across orchestration, data lakes/lakehouses, warehouses, data quality, real-time systems, and LLM app libraries — useful when you need to design pipelines that feed ML models or productionize inference.
  • Lightweight, link-first format: instead of reinventing tutorials, it points to canonical guides, whitepapers, and projects, lowering the time-to-resource for concrete hands-on practice and cohort bootcamps.
Who it's for and tradeoffs

Great fit if you are building a study plan to transition into data engineering or MLOps, preparing for interviews, or assembling a list of practical tools and projects to learn by doing. Look elsewhere if you need an opinionated, end-to-end code library or turnkey software package — the handbook is a curated index and learning guide, not a maintained SDK. Content quality depends on linked upstream sources and contributor maintenance, so verify versions for production decisions.

Information

  • Websitegithub.com
  • OrganizationsDataExpert-io, DataExpert.io
  • Published date2023/11/19

Categories

More Items

GitHub
AI Train2025

Provides a modular full-stack reinforcement learning stack to train and evaluate long-horizon, multi-turn tool-use LLM agents, including a performant trainer, a Tinker-compatible backend, agent orchestration, and Gymnasium-style environments for task design.

GitHub
AI Agent2026

Monitors and detects risky behavior in enterprise AI agents via high-fidelity telemetry, security benchmarking, and a two-tier detector. Comprises ADR Sensor, ADR-Bench, and ADR Detector; deployed in production at Uber and validated on public benchmarks.

GitHub
AI Infra2026

Rust library for fast PDF classification and position-aware text extraction that converts native-text PDFs to structured Markdown without OCR. Offers per-page OCR routing, multi-column and table detection, and Python/Node.js/WebAssembly bindings for low-latency local pipelines.