scd

smart substation connection and configuration software based on IEC 61850 protocal and SCD file. Email: 570503271@qq.com

Python Driver for Sensirion I²C Carbon Dioxid Sensor - SCD40, SCD41

Data Warehousing ETL Demo with Apache Iceberg on EMR Local Environment

How to manage SCD2 with Apache Hive 1.1 and HBase 1.2 w/o HiveQL UPDATE operation

Python ETL pineline craws data from Weather Forecast, transforms and loads to MySQL

File-format and runtime documentation for the PC (Steam) release of FINAL FANTASY TYPE-0 HD - archive format, PSP heritage, and the HexaDrive render layer

Source code for the python package: delta-hybrid-scd

End-to-end Databricks Lakeflow Spark Declarative Pipelines project for wine pricing and inventory data, using Medallion Architecture, PySpark, Auto Loader, data quality expectations, and SCD Type 1/Type 2 change tracking.

End-to-end batch data pipeline — Python, DuckDB, Airflow, Star Schema

A data pipeline on GCP Dataproc using Sqoop, HDFS, Hive, and PySpark to implement SCD Type 2 for an e-commerce use case. Tracks customer and product changes (e.g., address, price) and their impact on sales, demonstrating scalable data warehousing and processing.

End-to-End Azure Data Engineering Pipeline | Medallion Architecture | Databricks + Delta Lake + SCD + DLT

🌎 Batch lakehouse for 4.4M+ Chicago 311 service requests — Kimball star schema with documented SCD strategy, Write-Audit-Publish pattern on BigQuery/Iceberg, and dual dashboards for ops and SLA reporting.

Hands-on demo of Delta Lake Change Data Capture (CDC) using Change Data Feed — batch history reads + streaming CDC pipeline.

SCD (Types 0-6), upsert, surrogate key, checksum, and guarded Delta merge library for Databricks over Spark Connect — you shall not pass... bad data.

End-to-end PySpark ETL pipeline with incremental loading, partitioned Parquet storage and audit logging for production-style data processing.

A hands-on learning repository for dbt Core, focused on building data models using Databricks as the warehouse, Python for development, VS Code as the IDE, and GitHub for version control and collaboration.

General-ledger star schema in DuckDB with SCD Types 1, 2, 3, surfaced in Power BI. Synthetic data, zero cost, portable to Snowflake.

End-to-end business intelligence project using AWS S3, Snowflake, dbt, SQL, Python, and Plotly for retail sales performance and operational analysis.