Data Engineer – Azure & Cloudera Migration

About the role

We are looking for a skilled Data Engineer to lead the migration of large‑volume historical data from on‑premise Cloudera environments to Azure cloud services. The role focuses on building metadata‑driven pipelines, ensuring data quality, and supporting ongoing incremental loads.

Key responsibilities

• Design and implement Azure Data Factory pipelines for metadata‑driven data ingestion and migration.
• Execute large‑scale historical data extraction using Azure Data Box and load data into ADLS Gen2.
• Transform and process historical datasets in Azure Databricks (Spark/SQL) using Delta Lake.
• Orchestrate incremental data loads and ensure end‑to‑end data reconciliation and audit sign‑off.
• Collaborate with cross‑functional teams to validate migration outcomes and resolve data quality issues.

Required profile

• Hands‑on experience with Cloudera and cloud migration projects.
• Strong understanding of Azure data services, especially Data Factory and Databricks.
• Ability to work with large data volumes and ensure data integrity throughout migration.
• Detail‑oriented mindset for audit and reconciliation processes.

Required skills

• Azure Data Factory (ADF)
• Cloudera (on‑premise)
• Cloudera to Azure cloud migration
• Azure Databricks (Spark / SQL)
• Azure Data Box
• ADLS Gen2
• Delta Lake
• Data reconciliation and audit sign‑off

What we offer

• Opportunity to work on a high‑impact data migration project.
• Exposure to cutting‑edge Azure data technologies.
• Collaborative environment with skilled data professionals.