Python Data Engineer – AI & Generative AI Specialist

About the role

We are looking for a Python‑focused Data Engineer to design and operate scalable data pipelines that power AI and Generative AI applications. You will work with large datasets, build reusable modules, and ensure data quality for downstream machine‑learning models.

Key responsibilities

• Develop and maintain high‑performance data pipelines using Python and ETL best practices.
• Perform exploratory data analysis (EDA) to identify patterns and quality issues.
• Build reusable Python modules for data ingestion, transformation, and automation.
• Design and expose data services and APIs that support AI/ML and Generative AI workloads.
• Optimize workflows for scalability, reliability, and observability.
• Prepare and structure datasets for large language models (LLM) and retrieval‑augmented generation (RAG) systems.
• Integrate AI models from providers such as OpenAI, Anthropic, LLaMA, and Mistral into production pipelines.

Required profile

• Strong hands‑on experience with Python and core data‑science libraries (pandas, NumPy, Matplotlib, scikit‑learn, PyTorch).
• Proficiency in SQL and data modeling for both relational and NoSQL stores.
• Experience building end‑to‑end data‑to‑AI pipelines, including model integration and API development.
• Familiarity with AI orchestration frameworks (LangChain, AutoGen, CrewAI) and Generative AI use cases.
• Knowledge of cloud platforms (AWS, Azure, GCP) and distributed processing systems is a plus.

Required skills

• Python
• pandas
• NumPy
• Matplotlib
• scikit‑learn
• PyTorch
• SQL
• Data modeling
• ETL pipeline development
• API design
• OpenAI, Anthropic, LLaMA, Mistral
• LangChain, AutoGen, CrewAI
• AWS / Azure / GCP
• Vector databases & embedding retrieval
• Big data / distributed processing
• Monitoring & observability tools