Sr. Inference Service SRE & Automation Engineer

About the role

Bitdeer’s NeoCloud platform runs Infermesh, a managed inference/model‑as‑a‑service offering where customers call model endpoints via a public API. The role is responsible for end‑to‑end reliability of the inference serving path, aiming for a 99.95% SLA.

Key responsibilities

• Own availability of customer inference endpoints, including API uptime, request success rate, latency and SLA compliance.
• Manage the serving path components: API gateway, inference router, request scheduler, KV cache, inference engines and GPU model replicas.
• Operate GPU fleets (H200, B300, GB300, VR NVL72), monitor health signals (DCGM, Xid, ECC) and automate drain, cordon and replica replacement.
• Maintain the Kubernetes platform (HA control plane, etcd, CNI, CoreDNS, edge tiers) and shared dependencies such as PostgreSQL, Redis, Vault, object storage and ClickHouse.
• Define and instrument SLIs/SLOs, run error‑budget alerting and produce customer‑facing SLA reports.
• Lead incident management, on‑call rotation, post‑mortems and resilience validation through failover drills and fault injection.

Required profile

• 5+ years of SRE or production platform operations experience, with at least 2 years on a customer‑facing API service under a published SLA.
• Deep expertise in Kubernetes operations, multi‑cluster topologies and HA control‑plane management.
• Hands‑on experience serving large language models or similar inference workloads (vLLM, SGLang, TensorRT‑LLM, Triton, etc.).
• Proven track record with GPU fleet operations, including DCGM/Xid/ECC monitoring and NVLink/NCCL troubleshooting.

Required skills

• Kubernetes (HA control plane, etcd, CNI, ingress, multi‑cluster)
• GPU fleet management tools (DCGM, Xid, ECC)
• Inference runtimes: vLLM, SGLang, TensorRT‑LLM, Triton
• Observability stack: VictoriaMetrics, VictoriaLogs, Tempo, Pyroscope, Grafana, ClickHouse
• Stateful services: PostgreSQL, Redis, Vault, object storage
• Infrastructure as code: Terraform, Helm, GitOps (ArgoCD or Flux)
• Programming: Go, Python