Applied Scientist, Agent Evaluation & Adaptive Model Routing

About the role

Bitdeer AI Lab is seeking an Applied Scientist to design and operate evaluation and routing systems for large‑language‑model (LLM) agents. You will make agentic inference measurable, reliable, and adaptive across a range of tasks and operational constraints.

Key responsibilities

• Own and extend the LLM and agent evaluation pipeline, creating representative task suites and tracking metrics such as success, tool use, quality, cost, latency, and token consumption.
• Research and prototype adaptive model‑routing strategies, including rule‑based baselines, cascades, stage‑aware routing, uncertainty‑aware selection, and escalation/recovery policies.
• Transition promising methods from offline evaluation to traffic replay, shadow testing, and internal pilots, collaborating with MaaS and platform engineering for production integration.
• Collaborate on production gateway reliability, billing, access control, and SLA engineering.

Required profile

• Bachelor’s, Master’s or PhD in Computer Science, Machine Learning, Statistics, Electrical Engineering, or a related field.
• Hands‑on experience with LLM evaluation, agentic systems, applied machine learning, or adaptive inference.
• Strong programming ability in Python.

Required skills

• Python programming
• LLM evaluation
• Agentic systems
• Adaptive inference techniques
• Machine learning and statistical analysis

What we offer

• Opportunity to work in a frontier AI lab with global computing infrastructure.
• Access to cutting‑edge hardware and large‑scale data centers.
• Collaborative environment with cross‑functional engineering teams.