Enterprise Data Engineering Pipeline Ssis Pyspark

This is a full-stack enterprise data warehousing setup that combines SSIS for ETL orchestration, SQL Server with star schema design (fact and dimension tables), and PySpark for scaled analytics. You get the complete pipeline: ingesting raw CSVs, transforming through SSIS packages with lookup validations and error logging, loading into a dimensional model, then running Python audits for data quality checks and PySpark aggregations for heavy workloads. The source includes actual SQL DDL for dimensions and facts, SSIS component mappings, and Python scripts with SQLAlchemy connections. If you're building a classic Microsoft-stack data warehouse but need to scale beyond what T-SQL handles comfortably, this shows how to wire it all together with concrete error handling and BI views.

Install to Claude Code

npx -y skills add aradotso/data-skills --skill enterprise-data-engineering-pipeline-ssis-pyspark --agent claude-code