Jesus De Lucas Rodriguez

Senior Big Data Engineer in Insurance Customer:

• Design Architecture for Ingestion and ETL processes (Raw/Silver/Golden): For building Fact and Dimension tables: Using PySpark in AWS Architecture department in Retail Customer:

• Design Architecture for integration between Cloudera 7.1.7 SP2 and Google Cloud:

• Generate and integrate connectors: PubSub Lite, BigTable and BigQuery

• Technical Leader in Data migration from Cloudera Cluster ETL to Google Cloud BigQuery. (Scala/Spark)

• Implement tools/libraries transversal for all new projects:

• Loader/Extract files (from Cloudera to Big Query) using Spark/Scala

• Data quality feature to track process/governance in Cloudera cluster (Scala)

• HDFS compactor/Kudu Tables Optimizer/Extractor Loader Hive and Kudu Tables using Spark/Scala

• Implement TDD architecture for new projects (Maven/Scala) and Jenkins pipeline implementation. Big Data Solution for data migration to cloud infrastructure in Bank Customer:

• ETLS for data migration (Scala/Spark/AWS-EMR)

• Improvements on migration Framework (Scala) Design and implement Lambda Architectures for retail customer (Cloudera) Teach and Mentor A1 employees: AWS Course, GitHub, Spark/Scala New project offers contribution: Design Cloud Datalake using AWS CloudFormation

Test Based deployment Methodology contributor for the start of new projects

Big data solutions in some clients: Banking, Energy and Education On premise and cloud architectures (AWS) Azure: Ingestion with DataFactory and DataLakeStorage Gen1 Processing: Spark, Scala, Hive Impala Deployment: On premise Cluster (Cloudera) and Kubernetes

Big Data Training course and projects in Banking and transport: Spark/Scala, Spark Streaming con Kafka Supra Framework (Santander) Continuous delivery: Jenkins, Deployment Environment: Cloudera