Skip to content
View LiliValGo's full-sized avatar
🎯
Focusing
🎯
Focusing

Block or report LiliValGo

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
LiliValGo/README.md

Hi, I'm Lili Valencia Gonzalez

Data Engineer building batch and streaming pipelines on GCP and AWS, with a focus on data quality, reproducible infrastructure, and turning messy real-world data into decision-ready datasets.


What I work on

  • Pipelines & architecture — medallion / lakehouse patterns (Bronze → Silver → Gold), batch and streaming
  • Transformations — SQL and dbt models that run the same locally (DuckDB) and in the cloud (BigQuery)
  • Data quality — validation rules, confidence scoring, and tests that run in CI
  • Infrastructure as code — Terraform, containers, and serverless (Cloud Run, AWS Lambda)

Stack

Python SQL dbt DuckDB GCP BigQuery AWS Kafka Spark Docker Terraform PostgreSQL


Featured projects

Project What it does Stack
gcp-data-engineer-demand-pipeline Batch pipeline that tracks data-engineering job demand in LATAM. Medallion architecture, dbt models, DuckDB locally and BigQuery + Cloud Run on GCP, plus an MCP server for LLM querying. Python · dbt · DuckDB · BigQuery · Cloud Run · Terraform
SmartCity Real-time smart-city streaming pipeline for vehicle, GPS, weather and traffic data, stored as Parquet. Kafka · Spark · S3 · Docker
ETL-Pipeline-Environmental-Crimes ETL from a public API to PostgreSQL and S3, with structured logging, unit tests and serverless ingestion. Python · pandas · AWS Lambda · S3 · PostgreSQL
NLP-for-IPCC-Climate-Reports Web scraping, PDF processing and NLP to extract and analyze IPCC climate reports. Python · spaCy · pandas
AuriData Compact, containerized ETL system for historical loads with strict validation. Python · Docker
IRCA-Analysis-System CLI that computes water-quality risk (IRCA) from geolocated records. Rust · DuckDB

Community

  • Organizer at Google Developers Group (GDG)
  • Regular participant in data-centric hackathons, using my environmental engineering background to extract insights from environmental and climate data

Find me

LinkedIn · X

Pinned Loading

  1. gcp-data-engineer-demand-pipeline gcp-data-engineer-demand-pipeline Public

    Batch pipeline tracking data-engineering job demand in LATAM — medallion architecture, dbt, DuckDB/BigQuery, Cloud Run, MCP server.

    Python

  2. NLP-for-IPCC-Climate-Reports NLP-for-IPCC-Climate-Reports Public

    This project combines web scraping, PDF processing, and Natural Language Processing (NLP) to extract and analyze IPCC climate reports. It automates downloading PDFs, processes file validation, and …

    Jupyter Notebook 1

  3. SmartCity SmartCity Public

    Simulates a real-time Smart City data pipeline with Kafka, Apache Spark, and S3. Streams and processes vehicle, GPS, weather, traffic, and emergency data with Dockerized components and Parquet stor…

    Python 1

  4. ETL-Pipeline-Environmental-Crimes ETL-Pipeline-Environmental-Crimes Public

    ETL pipeline to analyze environmental crime data in Colombia, integrating data extraction from a public API, data transformation with pandas, and loading into PostgreSQL and AWS S3. Includes loggin…

    Python