Senior Data Engineer (Big Data Processing)

About the role

CSIT develops products that advance the national security interests of Singapore, used in areas such as counter-terrorism and computer network defence. It is looking for a senior data engineer to lead and build mission-critical products that identify, analyse and investigate threats. The role sits in a team responsible for mission-critical data analytics systems, contributing to high-performance data pipelines and systems.

Key responsibilities

• Design and build scalable, robust and maintainable data pipelines to ingest crucial data using cutting-edge platforms and technologies.
• Work with product managers, engineering managers and key stakeholders to deliver data products that meet business needs.
• Optimise large-scale data pipelines for low latency, high throughput and reliability.
• Ensure data integrity, quality and governance through robust design and operational monitoring.
• Troubleshoot and resolve complex issues in critical data pipelines.
• Apply best practices in data engineering, including testing, orchestration and monitoring of pipelines.
• Lead technical discussions with the team and stakeholders to shape the future of data products.
• Mentor junior data engineers and help build a strong data engineering culture.

Required profile

• Degree or professional qualification in computer science or equivalent.
• 5+ years of experience in data engineering, focused on building large-scale data pipelines.
• Proven experience writing optimised ETL code for large datasets.
• Proficiency in Scala or Java.
• Ability to communicate technical details clearly and good interpersonal, analytical and problem-solving skills.

Required skills

• Familiarity with Spark, Kafka, Hadoop, the Elastic stack and Kudu.
• Strong understanding of modern data architectures and data modelling paradigms.

CSIT is an agency under the Ministry of Defence (Singapore); only Singapore citizens will be considered. The company uses AI tools to support parts of the hiring process, but final hiring decisions are made by humans.