Senior Slurm Cluster & HPC Engineer

About the role

Bitdeer, a technology company for Bitcoin mining and AI cloud, is hiring a Senior Slurm Cluster & HPC Engineer in Singapore. You will design, deploy and operate production Slurm clusters on bare metal and VMs, manage multi-tenant GPU scheduling policies, and lead the implementation of Slurm-on-Kubernetes stacks.

Key responsibilities

• Design, deploy and operate production Slurm clusters on bare metal and VMs, with high availability, configless slurmd, SACK/MUNGE and JWT authentication, and rolling upgrades without losing running jobs.
• Model GPU fabric topology in topology.conf (rail-optimized InfiniBand/RoCE and NVLink domains) and validate placement with NCCL bandwidth and multi-node training tests.
• Own the account/association tree, partitions, QoS, fairshare, preemption, reservations and per-tenant TRES limits, enforcing fail-closed defaults.
• Lead the Slinky slurm-operator implementation and pilot slurm-bridge for co-scheduling Kubernetes workloads, documenting constraints such as exclusive whole-node allocation.
• Use Slurm cloud and power-save mechanisms with fleet automation to shift GPU nodes between batch training queues and Kubernetes inference capacity.
• Operate Pyxis/Enroot and OCI/containerd job paths, support MPI/PMIx, modules and Spack environments, and handle customer-supplied images.
• Build health checks, diagnostics and automatic drain/requeue, and own burn-in and acceptance testing for new racks.
• Deliver clusters via Terraform/Ansible, golden images and bare-metal provisioning (PXE, Redfish, IPMI).
• Instrument queues, allocation efficiency and GPU utilization into Prometheus/Grafana, configure accounting and billing weights, and reconcile GPU-hours with invoicing.
• Write runbooks and documentation, support enterprise customers, act as escalation point for incidents and mentor platform engineers.

Required profile

• 8+ years in HPC, systems or cloud infrastructure engineering, including 4+ years operating production Slurm clusters at 100+ GPU-node scale.
• Bare-metal and virtualized compute experience, including Terraform/Ansible automation.
• Knowledge of parallel and shared storage (Lustre, GPFS/Spectrum Scale, WEKA, VAST or NFS).
• Proficiency in Python and Bash; Go is a plus.
• Clear written and verbal English, able to work with enterprise customers and explain trade-offs to product, sales and executives.

Required skills

• Deep hands-on Slurm expertise (slurm.conf, gres.conf, topology.conf, cgroup.conf, accounting, slurmrestd, MUNGE/SACK, JWT authentication, live upgrades)
• NVIDIA drivers, Fabric Manager, DCGM, MIG, InfiniBand/RoCEv2, GPUDirect RDMA and NCCL tuning
• Production Kubernetes and at least one Slurm-on-Kubernetes stack (Slinky, CoreWeave SUNK or Nebius Soperator)
• Multi-tenant security discipline with fail-closed authorization

What we offer

• An inclusive, open-workspace environment with start-up spirit
• Fast-growing company with direct impact on the digital asset industry
• New projects, process development, personal accountability and autonomy
• Training, mentoring and welfare benefits