Descripción

Trayectoria iniciada como programador a los 15 años, especializado en el ecosistema Big Data desde sus primeras etapas. Más de 10 años de experiencia en arquitecturas de datos cloud, diseño de pipelines ETL/ELT complejos y migración de sistemas legacy hacia soluciones modernas utilizando Spark, Databricks, Snowflake y servicios cloud (AWS, Azure, GCP).

Experto en desarrollo de Data Lakes, optimización de costes y mejora del rendimiento mediante migraciones cloud nativas. Experiencia demostrada en sectores críticos como seguridad pública, retail y tecnología legal. Especialista en Apache Spark (PySpark, Scala), Snowflake, Databricks y metodologías ágiles.

PROYECTOS PERSONALES

• Análisis de Criminalidad en Londres - Pipeline ETL completo con Snowflake y visualización interactiva Streamlit

GitHub:

• Libro Técnico Publicado - Autor especializado en tecnologías de datos

Amazon:

Idiomas

Portugués
Bilingüe o nativo
Español
Competencia profesional completa
Inglés
Competencia profesional completa

Preferencias de lugar de trabajo

Solo teletrabajo

Lleva a cabo sus proyectos principalmente en remoto

Prodesp/DIPOL
Data Engineer / Data Scientist
febrero de 2022 - Hoy (4 años y 6 meses)
Developed routines in Databricks (PySpark) for processing and analyzing datasets related to Missing Persons and Death Records.
Built routines in Databricks (PySpark) to profile kidnappers.
Created web crawlers to find images of criminals online and compare them with a local database.
Established Data Factory pipelines to execute Databricks notebooks.
Designed routines for cross-referencing local data (SQL Server and Data Lake) with Public Safety Department APIs.
Developed Heat Maps for robbery and theft crimes using BigQuery and Looker.
Created scripts in PySpark (Dataproc) and Apache Beam (Dataflow) for migrating crime data from IBM DB2 to BigQuery for a heat map project.
Migrated data project from GCP to Snowflake
Created databases, schemas, and stages
Copied data from a Google Cloud Storage stage
Bulk loading from Google Cloud Storage
Created jobs in Snowpark
Developed dashboards using Streamlit
Google Cloud Platform (GCP) Dataproc Dataflow Databricks Azure Datalake
Smartfit
Data Engineer
mayo de 2021 - febrero de 2022 (9 meses)
Developed routines to monitor Alteryx ETL jobs
Created migration tasks using DMS
Configured Kinesis Data Streams as a target for DMS
Restructured Data Lake Zones
Migrated legacy jobs in Glue to Spark (PySpark) running on EKS
Migrated routines from EMR to Spark on EKS (Fargate)
Implemented pipelines using Airflow (MWAA)
Developed ETL routines with Spark (PySpark) on EKS for loading into parquet (S3/Athena) and Redshift
Conducted a PoC for migrating Tableau to Elasticsearch/Kibana
Migrated data project from AWS to Snowflake
Created databases, schemas, and stages
Copied data from S3 into staging table
Loaded data from staging table to target table
Created jobs in Snowpark
Developed dashboards using Streamlit
JBS/Seara (Project-Based)
Data Engineer
febrero de 2021 - mayo de 2021 (3 meses)
Implemented data pipelines with Composer (Airflow).
Developed ETL routines with PySpark (Dataproc) for loading datasets into BigQuery.
Migrated Oracle databases to BigQuery.
Developed web crawlers (Web scraping) for extracting data from government portals.
Dataproc PySpark Oracle Database BigQuery Airflow