Senior LLM Inference Performance & Evaluation Engineer

About the role

Bitdeer is seeking a Senior LLM Inference Performance & Evaluation Engineer to lead the development of benchmark pipelines, model launch gates, and performance monitoring for large language models. You will work closely with runtime engineers and SRE teams to ensure high‑quality, low‑latency AI services for our global customers.

Key responsibilities

• Design and implement benchmark pipelines measuring TTFT, inter‑token latency, throughput, request latency, concurrency, error rates, and GPU utilization.
• Create launch gates for OpenAI/Anthropic compatibility, streaming, tool calling, multimodal and long‑context scenarios.
• Maintain representative workloads using synthetic, replayed, and customer‑like traffic for steady‑state and burst conditions.
• Compare model, runtime, and provider options and publish recommendations for routing, fallback, pricing, and capacity.
• Automate regression detection in CI/CD and staging environments.
• Collaborate with runtime engineers to identify bottlenecks and with SRE to translate benchmarks into SLOs and alerts.

Required profile

• 5+ years of experience in ML infrastructure, performance engineering, model evaluation, QA automation, or backend testing for production systems.
• Proven ability to design statistically meaningful tests and communicate trade‑offs between quality, latency, reliability, cost, and user experience.
• Experience building dashboards, reports, and release gates for engineering and product stakeholders.

Required skills

• Python
• Go (optional but beneficial)
• OpenAI and Anthropic APIs
• vLLM, Dynamo, SGLang, Triton‑style servers
• Kubernetes
• CI/CD pipelines

What we offer

• A culture that values authenticity, diversity, and inclusive collaboration.
• Open workspaces with a fast‑growing start‑up spirit.
• Opportunities to impact the digital asset and AI cloud industries.
• Direct involvement in innovative projects and networking with industry pioneers.