Realtime Cinema Data Engineering Pipeline

This is a full-stack streaming pipeline tutorial that wires together Kafka, Postgres, Airflow, and Streamlit to process cinema transaction events through a bronze/silver/gold medallion architecture. You get a Faker-based event generator, a Kafka consumer that dumps JSON into Postgres, Airflow DAGs that normalize the data into 3NF tables, and a Streamlit dashboard with Plotly charts. It's designed to handle 1M+ events and teaches you how the pieces fit together in a production-style setup. The Docker Compose file spins up the infrastructure, but you'll need to wait a few minutes for Airflow to initialize before running the pipeline. Good for learning streaming ETL patterns without the AWS bill.

Install to Claude Code

npx -y skills add aradotso/data-skills --skill realtime-cinema-data-engineering-pipeline --agent claude-code