Senior AI Data Engineer

About the role

Singtel has launched a new business unit, AIDA (Artificial Intelligence & Data Analytics), to drive its AI transformation. As a Senior AI Data Engineer you will design, build and operate scalable data ingestion, transformation and serving capabilities on a modern hybrid cloud data platform, keeping solutions reliable, secure and reusable.

Key responsibilities

• Design, build, optimise and maintain batch and streaming data pipelines using Databricks and Kafka, with observability and alignment to enterprise data architecture.
• Perform data transformation and cleansing with PySpark or SQL, and monitor and troubleshoot data workflows.
• Integrate data from files, APIs, databases and streaming sources, defining ingestion patterns and delivery timelines with source and consuming teams.
• Lead knowledge base and retrieval-augmented generation (RAG) solution stacks, including storage, lifecycle management, embedding and vectorisation.
• Maintain metadata and pipeline documentation, and own production readiness including incident triage, root-cause analysis and release coordination.
• Integrate pipelines with Microsoft Fabric, Databricks, Delta Lake and other platform components.
• Apply data governance, security, access control and operational risk policies, and contribute to automation via version control and CI/CD.

Required profile

• Bachelor's degree in Computer Science, Engineering or a related field.
• 5-8 years of experience in data engineering, data platform engineering or cloud-scale analytics, owning production pipelines and platform components.
• Able to work independently and lead technical discussions with engineers, architects, product owners and business stakeholders.
• Strong documentation and communication skills.

Required skills

• Hands-on Python and SQL for data transformation and validation.
• Familiarity with Apache Spark (PySpark) and large-scale data processing.
• Experience implementing knowledge base and RAG solutions for agentic AI use cases.
• Understanding of enterprise data architecture, cloud security, access control, CI/CD, release management and production operations.