Data Engineering Study Material

This is a reference repository packed with Python examples covering the data engineering stack from ETL pipelines to Airflow DAGs. You'll find practical code for building pipelines with pandas and SQLAlchemy, data quality checks using Great Expectations, and orchestration patterns for AWS, GCP, and Azure. It's organized by topic (fundamentals, streaming, batch processing, cloud platforms) and includes interview prep materials. The examples are realistic enough to adapt for actual projects, like the ETL pipeline class with proper logging and error handling or the Airflow DAG with S3 to Redshift transfers. Best used as a quick reference when implementing pipelines or brushing up before interviews rather than as a tutorial to read front to back.

Install to Claude Code

npx -y skills add aradotso/data-skills --skill data-engineering-study-material --agent claude-code