Data Engineer / Senior Data Engineer (Python, PySpark, SQL, Databricks)

About the role

A Data Engineer or Senior Data Engineer is sought in Singapore, working from the office. The role focuses on designing, building and maintaining robust data pipelines and architectures, ensuring the reliability, scalability and performance of the data infrastructure that supports business intelligence and analytics.

Required profile

• At least 5 years of experience designing, building and maintaining data pipelines and architectures.
• Ability to review technical designs and code, set engineering standards and guide less-experienced engineers.

Required skills

• Strong Python, PySpark and SQL, including distributed processing, complex transformations, query optimisation, debugging and automated testing.
• Databricks, Apache Spark and Delta Lake for production batch or streaming workloads.
• AWS data engineering: S3, Glue, Lake Formation, IAM, KMS, Lambda, Step Functions, Event Bridge, CloudWatch, Redshift and MSK/Kinesis.
• Lakehouse or data-lake architecture design, including layered data patterns, dimensional or domain modelling and curated datasets.
• Incremental ingestion, change data capture, idempotency, late-arriving data, backfills, replay and recovery.
• Data quality, schema validation, reconciliation, metadata and lineage as part of production pipelines.
• Workflow orchestration, Git-based development, CI/CD, infrastructure as code (e.g. Terraform) and environment promotion controls.
• Pipeline observability, dashboards, alerting, runbooks, incident diagnosis and service recovery.
• Cloud security basics: least privilege, network boundaries, encryption, secret management, data classification and audit logging.