Data Engineer (Content Processing)

About the role

CSIT develops cutting‑edge products to safeguard Singapore's national security interests. We are seeking Data Engineers to join a specialized team operating at the heart of our mission, transforming hundreds of thousands of documents daily through large‑scale pipelines. In this role, you will go beyond traditional data engineering; you will architect intelligent systems that ingest unstructured data, generate vector embeddings, and unlock semantic search capabilities to uncover critical insights from massive datasets.

Key responsibilities

• Architect, design, and maintain high‑throughput pipelines that ingest, route, and process vast volumes of unstructured data, integrating AI models for automated classification and extraction.
• Develop and optimize transformation logic, including chunking strategies and generation of high‑quality vector embeddings for downstream retrieval.
• Build robust hybrid indexing systems that combine keyword search with vector‑based semantic search for instant, precise insights.
• Engineer scalable video analysis and image recognition solutions, extracting actionable metadata from diverse media formats.
• Uphold rigorous data quality and pipeline reliability standards while championing best practices in code quality, testing, and observability.
• Collaborate with frontend engineers, data scientists, and product managers to deliver high‑performance data services for national security operations.

Required profile

• Background in Computer Science or a related field with proven hands‑on experience in data engineering or software development.
• Experience building scalable ETL/ELT pipelines to process, transform, and validate large datasets.
• Strong understanding of data quality principles, pipeline reliability patterns, and ability to communicate complex technical concepts clearly.

Required skills

• Exposure to vector databases, text embedding techniques, LLM integration, or semantic search frameworks.
• Familiarity with container‑based environments such as Docker and Kubernetes, and modern CI/CD practices.
• Experience with multimedia processing tools or computer vision libraries.
• Background handling high‑volume, unstructured data in cloud‑native environments.

What we offer

• Work that matters: pipelines directly support Singapore's national security by turning raw data into actionable insights.
• Cutting‑edge tech: hands‑on experience deploying AI solutions and modern cloud‑native data technologies at scale.
• Collaborative culture: a talented team with a strong engineering ethos and a commitment to real work‑life balance.
• Room to grow: investment in engineers, fostering innovation at the intersection of data engineering and AI.