Product Manager, Evaluation & Data (AI)

About the role

Rakuten Asia Pte. Ltd., the regional headquarters for Asia based in Singapore's Central Business District, is hiring a Product Manager for Evaluation and Data within its AI business unit. The role makes the quality of AI products measurable, repeatable and actionable by owning the evaluation strategy, process and platform so teams can define success, prevent regressions, make confident release decisions and continuously improve AI experiences.

Key responsibilities

• Define success for AI products and turn requirements, user needs and risks into measurable criteria
• Select grading methods, calibrate automated graders with human judgment and review transcripts and failures
• Define the platform vision, roadmap and requirements with engineering and data teams
• Prioritise dataset management, model/prompt comparison, automated and human grading, multi-turn evaluation, trace review, dashboards and release integration
• Maintain balanced evaluation suites, convert production issues into regression tests and improve coverage and accuracy
• Establish governance, enable team and expert contributions and link evaluations to production monitoring, A/B testing, feedback and business KPIs

Required profile

• 5+ years of product management experience in AI/ML, evaluation, data products or platforms; consumer AI experience preferred
• Excellent analytical skills and experience with large datasets
• Ability to define platform requirements and communicate model/prompt versioning, tracing, data pipelines and release integration
• Experience with A/B testing, production monitoring, user feedback and human evaluation
• Excellent written and verbal communication, able to present complex evaluation results clearly
• Proven ability to lead cross-functional product, engineering, data science and domain teams