Skip to content
#

incremental-models

Here are 13 public repositories matching this topic...

Pipewright: an end-to-end data pipeline for a D2C brand's CEO dashboard. Python ingestion (API + file drops) into a lake, a versioned data contract, dbt on DuckDB or PySpark with cross-track parity, a data-quality publish gate, Airflow, Streamlit, Docker, incremental models, backfills and SLA alerts. main = starter, solution = reference.

  • Updated Oct 7, 2026
  • Python

End-to-end batch data platform over 10M food-delivery orders: S3 → Snowflake (medallion) → dbt → Airflow, plus an LLM layer that turns 300K free-text reviews into tested sentiment/topic columns. 17 dbt models, 80 data tests, RAG and text-to-SQL apps.

  • Updated Oct 5, 2026
  • Python

Metadata-driven end-to-end Airbnb data engineering pipeline built with dbt, Snowflake, and AWS, using Bronze-Silver-Gold architecture, incremental transformations, Jinja macros, ephemeral models, SCD snapshots, data testing, and dimensional fact modeling.

  • Updated Aug 24, 2026
  • Python

End-to-end ELT pipeline on 30M+ real NYC yellow taxi trips. Airflow ingests monthly Parquet files into PostgreSQL, dbt transforms them with incremental models, tests and freshness checks, and Metabase serves demand, tip and airport revenue dashboards.

  • Updated Nov 20, 2026
  • Python

Modern data stack reference: dbt + BigQuery + Airflow (Cloud Composer) with medallion layering, SCD2 snapshots, exposures, freshness SLAs, and 45× cost reduction via partition + cluster + incremental tuning.

  • Updated Apr 23, 2026
  • Python

Add this topic to your repo

To associate your repository with the incremental-models topic, visit your repo's landing page and select "manage topics."

Learn more