Lead Data Engineer

About the role

Singapore Airlines is looking for a Lead Data Engineer, a senior software developer with strong software engineering skills. The role builds custom open-source-based data ingestion and MLOps platforms, handling large or near-real-time datasets, maintaining high data quality and automating pipelines so they are robust and need less human intervention.

Key responsibilities

• Design and develop data engineering services using distributed databases (relational, columnar, graph, in-memory), Apache Airflow and Kafka, Kinesis and Spark
• Build MLOps production pipelines and support data scientists and ML engineers in deploying models at scale on cloud and on-premises GPU and CPU instances
• Design data models for high-volume, near-real-time and batch data, and build idempotent, atomic pipelines that are fault tolerant
• Develop intuitive, highly automated, self-service data platform functions for business users
• Design, build and operate scalable data pipelines on Databricks
• Champion next-generation data-ingestion technologies and give technical guidance on cloud architecture for data projects

Required profile

• BS in Computer Science or a related discipline; advanced degrees (PhD, MS) highly desirable
• 5+ years of industry experience in the listed technical areas
• Prior experience in modern software development (web frontend UI, backend API microservices, CI/CD, Scrum/Kanban)

Required skills

• Advanced Python, data structures and algorithms
• Open-source data ingestion platforms and data pipelines
• AWS (S3, RedShift, Elasticache, PostgreSQL, EMR) and Databricks (cluster management, AI Agents, job orchestration)
• Object-oriented or functional programming (Python, Java, Scala or C#)