Senior Data Engineer – PySpark, Databricks, AWS

About the role

We are looking for a Senior Data Engineer to design, build and maintain high‑performance data pipelines across our Singapore operations. You will work with modern cloud and on‑premise technologies to transform large volumes of structured and semi‑structured data.

Key responsibilities

• Design, develop and operate scalable ETL/ELT pipelines using Databricks, PySpark and Apache Spark.
• Implement data transformation logic in SQL and Python, ensuring performance, reliability and data accuracy.
• Build and maintain curated data layers with Delta Lake and Delta Live Tables.
• Integrate data from Oracle, IBM DB2 and other relational sources, optimizing queries and load processes.
• Develop and support AWS‑based data services, including storage, processing, monitoring and security.
• Optimize Spark jobs, SQL queries and workflow performance to reduce processing time.
• Monitor production pipelines, troubleshoot failures and ensure timely resolution.
• Collaborate with business and technical teams to translate data requirements into scalable solutions.

Required profile

• 5+ years of experience in data engineering, ETL development or data integration.
• Strong hands‑on experience with Databricks, PySpark and Apache Spark.
• Proven ability to build complex ETL/ELT pipelines and data models.
• Solid SQL expertise, including complex queries, joins, aggregations and performance tuning.
• Experience with AWS data services and cloud‑based processing.
• Familiarity with enterprise ETL tools such as IBM DataStage or Informatica PowerCenter.

Required skills

• PySpark
• Apache Spark
• Databricks
• Delta Lake
• Delta Live Tables
• SQL
• Python
• AWS (Lambda, Redshift, IAM, CloudWatch, Glue, EC2)
• Oracle
• IBM DB2
• IBM DataStage
• Informatica PowerCenter