Data Engineering Medallion Pipeline

This is a complete local data engineering stack that implements medallion architecture using Docker Compose. You get MinIO for object storage, Airbyte for ingestion, PostgreSQL as your warehouse, DBT for transformations across bronze/silver/gold layers, and Airflow for orchestration. The repo includes a Makefile with commands for everything from `make dbt-run` to `make trigger-dag`, plus Grafana dashboards for monitoring. It's genuinely end-to-end, meaning you can clone it, run `make setup && make start`, and have a working ELT pipeline with data quality tests and SCD Type 2 snapshots. Best for learning the full data engineering workflow locally or prototyping pipelines before moving to cloud infrastructure. The DBT models show realistic patterns like deduplication and profit margin calculations.

Install to Claude Code

npx -y skills add aradotso/data-skills --skill data-engineering-medallion-pipeline --agent claude-code