Data Engineer

About the role

Thoughtworks is looking for a Data Engineer to join its Singapore team. You will work on cutting‑edge big‑data tools and frameworks, building and maintaining data pipelines that solve complex client problems.

Key responsibilities

• Design and implement intricate data processing pipelines in collaboration with team members.
• Partner with data scientists to create scalable implementations of machine‑learning models.
• Write clean, test‑driven code and use continuous delivery practices to deploy, support, and operate pipelines.
• Apply standard big‑data models and develop data models for at least one modeling technique.
• Integrate data‑quality checks into daily development work.

Required profile

• 3+ years of experience writing high‑quality Python code.
• Strong understanding of cloud platforms (AWS, GCP or Azure); Databricks experience is a plus.
• Hands‑on experience with data modeling and modern data‑engineering tools.
• Experience building production‑grade data pipelines on distributed storage and processing platforms.
• Solid knowledge of DevOps practices, CI/CD (GitLab/GitHub) and Infrastructure as Code (Terraform).
• Familiarity with data visualization, data governance, security and privacy considerations.
• Experience with SQL and NoSQL databases.
• Comfortable working in Agile, Lean or Continuous Delivery environments.

Required skills

• Python
• AWS
• GCP
• Azure
• Databricks
• Data modeling
• Distributed storage platforms
• Distributed processing platforms
• CI/CD
• GitLab
• GitHub
• Terraform
• Data visualization
• Data governance
• Security
• Privacy
• SQL
• NoSQL
• Agile
• Lean
• Continuous Delivery