Staff Platform Engineer - Big Data Platform Management

About the role

CSIT develops cutting‑edge products to protect Singapore’s national security. We are seeking a senior Staff Platform Engineer to lead the design, implementation and operation of our Big Data Infrastructure platform.

Key responsibilities

• Design, build and manage the Big Data platform, including Hadoop, Spark, NoSQL databases and data lakes.
• Operate and optimise large‑scale clusters, handling monitoring, troubleshooting, job scheduling and resource allocation.
• Implement storage solutions such as HDFS, S3 and other object stores, ensuring high availability and performance.
• Enforce security and governance, managing access controls, encryption and regular audits.
• Collaborate with data engineers and scientists to optimise application performance and provide technical guidance.
• Plan capacity, forecast needs and oversee procurement and deployment of new hardware and software.

Required profile

• Degree in Computer Science, Engineering or a related field.
• At least 8 years of experience managing Big Data platforms and leading engineering teams.
• Proven experience with Spark, Iceberg, Trino, Elastic Stack, NiFi, Kafka and Airflow.
• Strong background in networking, firewalls, DNS, routing and subnetting.
• Experience with agile development methodologies and version control (Git).

Required skills

• Spark, Hadoop, Iceberg, Trino, Elastic Stack, NiFi, Kafka, Airflow
• S3, NFS, SMB, HDFS storage protocols
• SSL/TLS encryption, data‑at‑rest encryption, OIDC/OAuth2 authentication
• Bash, Python, Scala, Java scripting
• Kubernetes orchestration and CI/CD integration
• Git version control

What we offer

• Purposeful work that contributes to national security
• Opportunity to work with top engineers and modern technology stacks
• Strong engineering culture with excellent work‑life balance
• Focus on engineering and operational excellence
• Support for innovation and continuous learning
• Collaborative family‑like environment