Senior Data Engineer · Azure Databricks · Snowflake · AWS · PySpark · Kafka · Dallas, TX (open to remote)
12+ years in data, building batch and streaming data platforms for financial services, healthcare, and retail: PySpark and Databricks pipelines, Snowflake ingestion and modeling, Kafka streaming, dimensional models, data quality, and CI/CD for data code.
These are representative projects written from scratch with synthetic data. They show patterns I use at work; they contain no employer or client code or data.
| Project | What it shows |
|---|---|
| healthcare-claims-lakehouse-pyspark | Medallion bronze/silver/gold pipeline in PySpark, data quality rules with a quarantine table, star schema and KPI mart, Databricks/Delta-ready |
| realtime-fraud-detection-kafka | Kafka producer/consumer with keyed, stateful fraud rules, at-least-once delivery, offline precision/recall replay |
| snowflake-incremental-cdc-pipeline | Snowpipe → Streams → Tasks → SCD Type 2, RBAC and resource monitors, SCD2 SQL unit-tested on DuckDB |
| sas-to-pyspark-migration-patterns | Seven SAS DATA step / PROC patterns migrated to PySpark with parity tests and a reconciliation helper |
Python · SQL · PySpark · Spark SQL · Azure Databricks · Azure Data Factory · Azure Synapse · ADLS Gen2 · AWS Glue · EMR · S3 · Redshift · Snowflake · Kafka · Airflow · PostgreSQL · Great Expectations · GitLab CI/CD · Azure DevOps · Power BI