Databricks Data Engineer

About the role

The Data Engineer will design, develop and maintain scalable data pipelines on Databricks and cloud platforms. You will work closely with analytics, product and infrastructure teams to deliver reliable data solutions that support reporting, analytics and machine‑learning workloads.

Key responsibilities

• Develop and maintain ETL pipelines for centralized storage such as Delta Lake.
• Integrate data from databases, APIs, log files, streaming platforms and external providers.
• Implement data transformation routines to clean, normalize and aggregate complex datasets.
• Collaborate with analytics and product leaders to design and operationalise pipelines.
• Monitor pipelines, create alerts and automated error‑handling mechanisms.
• Support SIT, data validation, UAT and production rollout.

Required profile

• Certification in Databricks.
• 3+ years of data‑engineering experience building scalable pipelines.
• Strong background in data modelling and distributed computing architectures.
• Experience working in Scrum teams and applying Agile practices.
• Problem‑solving mindset with a collaborative approach.

Required skills

• Databricks, Delta Lake
• PySpark, Spark SQL, Databricks notebooks/jobs
• Azure Data Factory (ADF) or Airflow for orchestration
• SQL (including window functions and performance tuning)
• Azure cloud platform (AWS experience a plus)
• Git, CI/CD pipelines, DevOps concepts
• Streaming technologies such as Apache Kafka, Apache Flink, AWS Kinesis