Marcelo Ribeiro

Responsible for the maintenance and development of new datasets in Google Cloud BigQuery Data Warehouse, leveraging Terraform for infrastructure management. Additionally, I designed robust data pipelines, ensuring efficient data flow and storage. Proficient in managing and scaling datasets within Google Cloud BigQuery. Expert in designing and implementing data pipelines. Advanced skills in Python, with practical applications in data handling and process automation. Experienced in using Apache Airflow for data orchestration, Google PubSub for real-time data streaming, and PySpark for data processing and integration. Key Achievements Real-Time ETL Development: Engineered a serverless ETL process to extract data from the EVA POS REST API using Google Cloud Functions and Python. This process feeds data directly into BigQuery, providing the business with access to real-time analytics. Data Integration Using PySpark: Utilised PySpark to integrate JSON data efficiently into BigQuery, handling a massive volume of approximately 20 billion rows (8TB), ensuring speed and data integrity during the data processing stage. Data Orchestration: Implemented Dataform to streamline the movement and management of data across various datasets in BigQuery, enhancing the overall workflow and data utilisation. Real-Time Data Streaming: Set up and managed Google PubSub for real-time data streaming, enabling immediate data availability and responsiveness for critical business operations.

• Aid in the scalability, security and repeatability of data pipelines.

• Development of Data Models and ETL using Python, SQLAlchemy and AWS Lambda.

• Providing maintenance to MS SQL Server using T-SQL, SSIS and SSRS.

• Full-stack development of web applications using Python Flask, Bootstrap and JavaScript.

• Integration between e-commerce tools Shopify and NetSuite using REST and GraphQL APIs.

• Implementation of the same framework used by EMEA to handle and manage personal and sensitive data according to the Brazilian version of GDPR (LGPD).

• Ensure the data protection of all entities related to the business.

• Support the operations and projects teams to implement new solutions and applications.