Lead Data Engineer

About the role

Singapore Airlines is looking for a Lead Data Engineer, a senior software developer with strong software engineering skills. The role builds custom open-source-based data ingestion and MLOps platforms, handling large or near-real-time datasets, maintaining high data quality and automating pipelines so they are robust and need less human intervention.

Key responsibilities

• Design and develop data engineering services using distributed databases (relational, columnar, graph, in-memory), Apache Airflow orchestration and Kafka, Kinesis and Spark
• Build MLOps production pipelines and help data scientists and ML engineers deploy models at scale on cloud and on-premises GPU and CPU instances
• Design data models for high-volume, near-real-time and batch data, and build fault-tolerant, idempotent pipelines
• Build automated, self-service data platform functions for business users
• Design, build and operate scalable data pipelines on Databricks
• Evaluate and champion next-generation data-ingestion technologies, take part in project planning and give technical guidance on cloud architecture

Required profile

• BS in Computer Science or a related discipline; advanced degrees (PhD, MS) highly desirable
• 5+ years of industry experience in the listed technical areas
• Prior experience in modern software development (web frontend UI, backend API microservices, CI/CD, Scrum/Kanban)
• Strong grasp of object-oriented or functional programming (e.g. Python, Java, Scala, C#)

Required skills

• Advanced Python, data structures and algorithms
• Open-source data ingestion platforms and data pipelines (collection, warehousing, processing, analysis, monitoring, governance)
• AWS components and best practices, including S3, RedShift, Elasticache, PostgreSQL and EMR
• Databricks workspace, cluster management, AI Agent capabilities and job orchestration