Descripción

🚀 Senior Data Engineer | Scalable Cloud Pipelines • Data Streaming • Big Data Solutions

I’m a Spain-based Senior Data Engineer with a proven track record in designing, building, and optimizing large-scale data infrastructures across AWS, Databricks, and SQL ecosystems. My experience spans end-to-end ETL/ELT pipeline development, real-time data streaming, and big data analytics — always with a focus on performance, scalability, and business impact.

Over the past years, I have:

Built and maintained AWS-based ETL infrastructures using Apache Airflow on Linux EC2.
Designed data lakes with S3 + Athena and integrated RabbitMQ for high-volume streaming.
Led Databricks/Spark projects for large-scale transformations and analytics.
Deployed and managed Dockerized ML models in SageMaker for production.
Tuned SQL Server and Redshift environments for high-traffic analytics.
Mentored teams and delivered robust solutions under agile frameworks (Jira, ClickUp).

Core strengths:

Data Pipelining & Orchestration (Airflow, Spark, Kafka, Flink, dbt)
Cloud & Big Data Platforms (AWS, Databricks, Hadoop, Hive)
Data Streaming (RabbitMQ, real-time processing, concept drift research)
Database Optimization (SQL Server, PostgreSQL, MySQL, Redshift)
Automation & Integration (FastAPI, web scraping, APIs)

Why work with me:

✅ Deep hands-on expertise with modern data ecosystems

✅ Strong research background in streaming & ML data adaptation

✅ Flexible part-time/remote availability across EU/US time zones

✅ Proven ability to translate complex requirements into scalable solutions

Whether you need a high-performance pipeline, cloud migration, or streaming data platform, I can integrate quickly, deliver fast, and ensure your data works harder for your business.

📍 Based in Spain — Available for remote projects worldwide.

Idiomas

Inglés
Bilingüe o nativo
Alemán
Nociones

Preferencias de lugar de trabajo

Acepta trabajo presencial

Málaga (hasta 50 km)

Marbill Technologies
Senior Data Engineer
agosto de 2022 - Hoy (4 años)
Marbella, Málaga, Spain
• Designed and built a scalable AWS-based ETL infrastructure using Apache Airflow on a Linux EC2 instance.
• Integrated RabbitMQ for data streaming, handling sending and receiving data between services and pipelines.
• Led a project on Databricks, leveraging its collaborative environment and Spark-based processing for large-scale data transformations and analytics.
• Established a test environment mirroring production for development and validation of new pipelines.
• Monitored and optimized cloud infrastructure health, performance, and costs, ensuring efficient resource utilization.
• Managed and optimized SQL Server database performance to support high-traffic reporting and analytics.
• Built a data lake architecture using Athena and S3, serving data scientists, analysts, and data engineers.
• Developed, monitored, and optimized several Airflow DAGs to ingest, clean, transform, and process data from various sources (APIs, S3, MySQL Server, RabbitMQ).
• Stored processed data in S3, Redshift, and SQL Server, enabling analytics and reporting.
• Optimized existing Airflow DAGs and data pipelines for better performance and scalability.
• Worked closely with business and marketing teams to translate requirements into data pipelines that support reporting and decision-making.
• Assisted the Data Science team by setting up SageMaker instances, training models, and troubleshooting daily issues.
• Deployed Dockerized data science models on SageMaker for production use.
• Led and mentored a team of 3 data engineers, providing technical guidance and career development support.
• Scaled and optimized AWS infrastructure and data pipelines to support business growth and evolving performance needs.
• Utilized agile project management platforms such as ClickUp and Jira to coordinate tasks, manage sprints, and ensure timely delivery of data engineering solutions.
RabbitMQ Docker Apache Airflow Data Pipeline Architecture & Optimization AWS Cloud
Cyshield
Data Engineer
mayo de 2021 - julio de 2022 (1 año y 2 meses)
Cairo, Cairo Governorate, Egypt
• Worked with local servers to support multiple projects, including a search engine, a Netflix-like streaming platform, and an Optical Character Recognition (OCR) system.
• Assisted in data modeling for database tables to ensure optimal structure and performance.
• Developed web scraping pipelines using Apache Airflow to collect and process data.
• Built and deployed REST APIs using FastAPI to support application functionality and data access.
• Integrated Grafana for real-time monitoring and visualization of system performance and data pipelines.
• Managed and optimized the deployment of data pipelines, APIs, and databases (PostgreSQL) to ensure smooth operation.
Nile University
Data Engineer | Data Streaming ResearchAssistant
octubre de 2020 - octubre de 2022 (2 años)
Giza, El Omraniya, Giza Governorate, Egypt
• Spearheaded research on online machine learning and concept drift detection, evaluating and adapting drift detection methods for improved performance across diverse data stream scenarios.
• Developed and implemented an adaptive bucket dropping technique that significantly reduced memory consumption while maintaining drift detection accuracy.
• Designed and executed extensive experiments on both synthetic and real-world datasets to assess throughput, drift detection performance, and resource utilization.
• Main Author of the ACM conference paper, (DOI: 10.1145/3477314.3507074), addressing efficiency improvements in data stream analytics.
• Contributed as a co-author to a Springer publication (DOI: 10.1007/978-3-031-21595-7_4), advancing the understanding of concept drift adaptation techniques in online machine learning.
• Collaborated with interdisciplinary research teams, sharing insights and validating experimental methodologies to drive innovation in streaming analytics.
• Presented findings at academic conferences, contributing to the broader research community and influencing future studies in data stream processing.
• Served as a member of the judges committee for university research papers and undergraduate projects, evaluating academic work and fostering research innovation.