Data Engineer (AVP) - Data Warehouse & AI Knowledge Base

About the role

OCBC is looking for a Data Engineer (AVP) to build, maintain and improve data pipelines that power an enterprise-grade data warehouse and an AI knowledge base in a banking environment. You will work closely with senior engineers and the head of data engineering to turn structured and unstructured data into reliable datasets and services supporting risk management, customer engagement, fraud detection and intelligent automation. The role reports to the VP/ED, Data Engineering, Group data office.

Key responsibilities

• Build and maintain batch and streaming pipelines feeding data warehouse and lakehouse platforms (Cloudera, AWS Redshift, GCP BigQuery)
• Develop batch jobs with Spark, SQL, Python or Java, and support real-time streaming with Flink or similar
• Help build CDC pipelines (Debezium, Confluent, Fivetran) and ingestion pipelines from APIs, GA4 and other sources
• Integrate messaging and streaming with Pub/Sub and Kafka, and write clean, tested SQL and Python scripts
• Develop Airflow DAGs and orchestrate workflows following senior engineers' best practices
• Deploy workloads on Cloudera, GCP (Docker, Kubernetes, Cloud Run) and AWS equivalents, maintain CI/CD pipelines and use Terraform for infrastructure
• Build REST APIs and backend services with Python/Flask, and use Redis caching where needed
• Support AI knowledge base and RAG work: vector database pipelines, embedding generation, document chunking, and monitoring of semantic search quality
• Collaborate with senior engineers, AI teams and business stakeholders (Risk, Marketing, Operations), and take part in code reviews

Required profile

• Diploma, Bachelor's or Master's degree in computer science or a related field
• At least 5 years of experience in data engineering, data platforms or related roles
• Experience building data pipelines on modern architectures (data warehouse, lakehouse, batch and real-time processing)
• Experience in banking or financial services is a plus

Required skills

• Data warehouse: Cloudera, BigQuery, Redshift or similar
• Batch: Spark, SQL, ETL, Python, Map/Reduce
• Streaming: Flink or similar (good to have)
• CDC: Debezium, Confluent, Fivetran (good to have)
• Ingestion: APIs, GA4, Pub/Sub, Kafka, Python pipelines
• Orchestration: Airflow or equivalents
• Cloud and DevOps: GCP or AWS, basic Docker/Kubernetes/Cloud Run, CI/CD, Terraform
• Backend: Python, Flask, REST APIs; Redis a plus
• Preferred: vector databases, embeddings, semantic search, RAG, LLM/AI agent concepts

What we offer

• Competitive base salary and a suite of flexible benefits
• Learning and development opportunities
• Exposure to enterprise-scale data and AI initiatives across the OCBC Group
• A collaborative environment that values innovation and craftsmanship