Data Engineer (Content Processing)

About the role

CSIT develops products that safeguard Singapore's national security interests. It is looking for Data Engineers to join a team that processes hundreds of thousands of documents each day through large-scale pipelines. The role goes beyond traditional data engineering: you will build systems that ingest unstructured data, generate vector embeddings and enable semantic search so analysts can detect threats in areas such as cybersecurity.

Key responsibilities

• Architect, design and maintain high-throughput pipelines that ingest, route and process unstructured data, integrating AI models for automated classification and information extraction.
• Develop transformation logic that parses complex content, applies chunking strategies and generates vector embeddings for downstream retrieval.
• Build and support hybrid indexing systems combining keyword search with vector-based semantic search.
• Engineer scalable video analysis and image recognition solutions using computer vision models to extract metadata from media streams.
• Uphold data quality and pipeline reliability standards and promote code quality, testing and observability.
• Collaborate with frontend engineers, data scientists and product managers to deliver high-performance data services.

Required profile

• Degree or professional qualification in Computer Science or a related field, with hands-on experience in data engineering or software development.
• Experience building scalable ETL/ELT pipelines to process, transform and validate large datasets.
• Good grasp of data quality principles and pipeline reliability patterns, and clear communication of technical concepts.
• Nice to have: exposure to vector databases, text embedding, LLM integration or semantic search; Docker/Kubernetes and CI/CD; multimedia processing or computer vision tools; high-volume unstructured data in cloud-native environments.

As CSIT is an agency under the Ministry of Defence (Singapore), only Singapore citizens will be considered. The company may use AI tools to help review applications, and final hiring decisions are made by humans.