AI Consultant - Data Platform & Multi-Agent Framework

About the role

NCS, an AI tech services company and subsidiary of the Singtel Group, is hiring an autonomous AI Consultant in Singapore. You will design and build the AI harness, agent framework and reusable engineering components needed to run a modern data platform built on Apache Iceberg, Trino and Parquet. Beyond building data pipelines, you will set up the agentic execution framework that lets multiple specialized agents collaborate, maintain state, orchestrate workflows and deliver end-to-end data engineering capabilities.

Key responsibilities

• Design and implement the foundational agent framework: agent orchestration, workflow execution and state management, memory and context handling, tool and function calling, agent-to-agent communication, human approval workflows, logging and audit trails, error handling and recovery, security and access controls, observability and monitoring.
• Create specialized agents that discover Iceberg catalogs and schemas, profile datasets, generate metadata and lineage, ingest data from databases, APIs, files and streams, create Parquet datasets, register and maintain Iceberg tables, build Trino SQL, create transformation workflows and curated datasets, validate schemas and data integrity, detect anomalies, generate reconciliation reports, analyze Trino workloads, recommend partitioning strategies, optimize Iceberg tables and Parquet files, classify sensitive information, apply retention and compliance policies, monitor pipelines and agents, detect failures and bottlenecks, and generate alerts and operational insights.
• Deliver Iceberg table definitions and schemas, Trino SQL scripts and views, Python data pipeline code, data quality validation suites, metadata and lineage documentation, CI/CD deployment artifacts, monitoring and observability configurations, and technical documentation and runbooks.

Required profile

• Experience with Generative AI, Agentic AI or LLM-based solutions.
• Hands-on experience with LangChain and LangGraph.
• Experience designing and implementing AI harnesses or multi-agent frameworks.
• Proven ability to design, develop and operationalize enterprise-grade AI agents for data discovery, ingestion, transformation, optimization, governance and observability.

Required skills

• Python, including FastAPI, Pydantic and Pandas.
• Apache Iceberg, Trino and Parquet-based lakehouse architectures.
• Data ingestion, transformation and orchestration pipelines.
• Metadata management, data lineage and data governance.
• REST APIs, SDKs and reusable engineering frameworks.
• Observability, monitoring, tracing and operational support.
• Distributed systems, event-driven architectures and microservices.
• Familiarity with DuckDB, dbt and related data engineering tools.