Data Engineer – Digital Transformation & Data

About the role

We are looking for a senior Data Engineer to lead the design, development and optimisation of data pipelines within our Digital Transformation & Data (DT&D) team. The role will focus on building robust, scalable data solutions on Azure and Databricks, ensuring data quality, governance and performance across the enterprise.

Key responsibilities

• Design, develop and maintain data pipelines that extract, transform and load data from multiple business applications, databases and external sources.
• Implement data quality checks, validation rules and enrichment processes to guarantee accurate and complete data.
• Manage logical and physical data models, enforce data governance and security best practices.
• Build and maintain CI/CD pipelines for code deployment and cloud resource provisioning using Azure DevOps and Databricks LakeFlow.
• Optimise data processing workflows for performance, scalability and cost efficiency, including capacity planning and monitoring.
• Support AI solution deployment on the data platform and collaborate with business users to create reports and dashboards.

Required profile

• Bachelor’s degree in Computer Science, Data Science, Software Engineering or related field; Master’s preferred.
• Minimum ten years of experience in data management, integration, modelling and quality.
• At least four years of hands‑on experience designing data architectures on Azure and Databricks.
• Proven ability to work autonomously and guide business stakeholders on data‑driven initiatives.

Required skills

• Python (including PySpark) and SQL programming.
• Experience with Java or Scala (advantage).
• Strong knowledge of Azure cloud services and Databricks platform.
• Proficiency with Azure DevOps, CI/CD, version control and Databricks LakeFlow jobs.
• Familiarity with data catalogue tools such as Unity Catalog and Microsoft Purview.
• Experience with relational (SQL) and non‑relational (NoSQL) databases; Oracle knowledge is a plus.
• Knowledge of Power BI and Python visualisation libraries for reporting.