Senior Inference Runtime Engineer

About the role

Bitdeer is looking for a Senior Inference Runtime Engineer to lead the performance‑critical serving layer of its MaaS platform. You will make self‑hosted large language models faster, cheaper, and more stable by optimizing the runtime stack behind OpenAI‑ and Anthropic‑compatible APIs.

Key responsibilities

• Optimize prefill/decode scheduling, continuous batching, KV‑cache behavior, speculative decoding, long‑context serving, and streaming smoothness.
• Tune and operate vLLM/Dynamo/SGLang/TensorRT‑LLM‑style runtimes for latency, throughput, GPU utilization, and cost efficiency.
• Profile bottlenecks across GPU memory, HBM bandwidth, NCCL/network, tokenizer, frontend/proxy, and model worker paths.
• Lead high‑value model onboarding, selecting runtimes, parallelism strategies, quantization, context length, and rollback plans.
• Define runtime playbooks and safe defaults for reasoning, tool calling, multimodal, prompt cache, and provider‑specific parameters.
• Collaborate with SRE and performance engineers to turn benchmark findings into production improvements.

Required profile

• 6+ years of systems, ML infrastructure, or high‑performance backend engineering experience.
• Hands‑on experience with LLM serving runtimes such as vLLM, Dynamo, SGLang, TensorRT‑LLM, TGI, or Triton.
• Strong understanding of GPU memory, CUDA/NCCL basics, KV‑cache, batching, streaming, and distributed inference trade‑offs.
• Proficient in Go or Python and comfortable reading runtime source code, profiling traces, and production metrics.
• Experience operating production inference services with strict latency, availability, and cost targets.

Required skills

• vLLM
• Dynamo
• SGLang
• TensorRT‑LLM
• TGI
• Triton
• GPU memory management
• CUDA
• NCCL
• KV‑cache handling
• Batching and streaming techniques
• Distributed inference
• Go
• Python
• Performance profiling
• Production metrics analysis

What we offer

• A culture that values authenticity and diverse perspectives.
• An inclusive environment with open workspaces and a startup spirit.
• Fast‑growing company providing networking opportunities with industry pioneers.
• Direct impact on cutting‑edge AI cloud and Bitcoin mining technologies.