AI Data Engineer – Cloudera to AWS Migration
PERSOL SINGAPORE PTE. LTD. · Singapore
Job description
About the role
We are looking for an AI Data Engineer to lead the migration of Cloudera/Hadoop workloads to AWS and to build enterprise‑scale AI, machine‑learning and generative‑AI solutions. The role combines data‑engineering, model development and MLOps in a fast‑moving, cloud‑first environment.
Key responsibilities
- Design, develop and deploy AI, ML and Generative AI applications across hybrid and cloud platforms.
- Build and optimise data processing pipelines using Python, SQL, Spark and PySpark.
- Implement Retrieval‑Augmented Generation (RAG) solutions with embeddings, vector databases and large language models.
- Migrate and modernise Cloudera/Hadoop workloads to AWS services such as SageMaker, Glue, EMR, Athena, Redshift and S3.
- Develop end‑to‑end MLOps frameworks for CI/CD, model lifecycle management, monitoring and governance.
- Collaborate with data‑engineering, architecture and business teams to translate requirements into scalable AI solutions.
- Evaluate emerging GenAI and LLM technologies to enhance business capabilities.
Required profile
- Strong hands‑on experience with Python, SQL and Spark/PySpark.
- Deep knowledge of Cloudera/Hadoop ecosystems (HDFS, Hive, Impala, CDP).
- Proven experience migrating data workloads from Cloudera to AWS.
- Solid understanding of machine‑learning pipelines, feature engineering and model deployment.
- Familiarity with generative AI concepts such as RAG, embeddings and LLM integration.
Required skills
- Python, SQL, PySpark/Spark
- Cloudera, Hadoop, HDFS, Hive, Impala, CDP
- AWS services: S3, Glue, EMR, Athena, Redshift, SageMaker
- Machine‑learning: feature engineering, model training, evaluation, deployment
- Generative AI: RAG, embeddings, vector databases, LLMs
- MLOps: CI/CD, model lifecycle management, monitoring
- Optional: LangChain, Bedrock, OpenAI, Azure OpenAI, Pinecone, Weaviate, OpenSearch, Kubernetes
Questions fréquentes
Why are you reporting this job?
Explore further
Salaries, guides and searches in Singapore.
Salaries by job title
Apply in 30 seconds
Enter your email to apply. An account will be created automatically.
By continuing, you accept our terms of use.
Already have an account? Login
Published 1 month ago
Expires 1 week from now
54 views · 0 interested
Boost your chances
Upload your CV — we will match you with relevant openings.
Analyzing your CV...
PERSOL SINGAPORE PTE. LTD.
Singapore
Related job offers
-
Senior Director, Solutions Architecture, APAC
cohere Singapore -
Forward Deployed Engineer – Infrastructure Specialist
cohere Singapore -
Release Engineer - Data Plane Internal Tooling and Productivity
clickhouse Singapore -
Enterprise Solutions Architect - Greater China Region
clickhouse Singapore -
Senior Consulting Engineer
clickhouse Singapore