Open Source data engineering demo project using dbt, DuckDB, dlt, Dagster and Metabase.
-
Updated
Oct 28, 2026 - Python
Open Source data engineering demo project using dbt, DuckDB, dlt, Dagster and Metabase.
SCD2 implementation using pyspark
A modern banking data pipeline built with Dagster and DBT!
Fortune-500-grade banking analytics platform: OLTP -> medallion lakehouse -> Kimball star schema -> semantic layer -> 9-tab executive dashboard + 5 ML models (churn, fraud, segmentation, forecasting). Production-ready, governed, fully tested.
Governed ARR, retention, and AI-adoption metrics with SCD2 modeling, role-based access, and safe SQL compilation.
PostgreSQL service-desk DB for a residential complex: SCD2 dimension, validation triggers, partitioning, EXPLAIN demos and a seeded Faker data generator
This is a data engineering pipeline built on Databricks + Delta Lake + PySpark that ingests travel booking and customer master data, applies SCD Type 2 logic, and delivers analytics-ready tables. It includes data quality enforcement, dimension versioning, fact aggregation, and performance tuning.
Pipeline ETL MySQL en 3 couches - staging, modele en etoile avec SCD Type 2, marts analytiques. Orchestrateur Python, 18 tests de coherence inter-couches
Analytics data warehouse for a production livestock ERP — dbt + BigQuery, 5-layer architecture, DAMA-DMBOK governance implemented as executable tests.
PostgreSQL data warehouse for government citizen service analytics: Medallion layers, star schema with five SCD2 dimensions, and explicit handling of five source data-quality defects
PySpark bronze/silver/gold CDC pipeline with SCD Type 2, data-quality gates and idempotent runs
Modern data stack reference: dbt + BigQuery + Airflow (Cloud Composer) with medallion layering, SCD2 snapshots, exposures, freshness SLAs, and 45× cost reduction via partition + cluster + incremental tuning.
Production-style retail lakehouse implementing incremental ingestion, CDC, SCD Type 2, data quality, and dimensional modeling across Bronze, Silver, and Gold layers.
dbt patterns I use on every project — free shortlist + link to the full printable reference pack
Move data between databases, files, and streams from Python.
To associate your repository with the scd2 topic, visit your repo's landing page and select "manage topics."