Senior Software Engineer - LLM Inference Optimization

About the role

Rakuten Asia Pte. Ltd., the Asia regional headquarters of Rakuten, is hiring an LLM Inference Optimization Engineer within its AI & Data Division. The engineer focuses on maximizing the performance, efficiency and scalability of LLM inference workloads on Rakuten's GPU clusters, optimizing inference engines such as vLLM, SGLang and TensorRT-LLM along with their GPU kernels and runtimes.

Key responsibilities

• Profile, optimize and extend modern LLM inference engines to improve throughput, latency and GPU utilization.
• Design and implement inference-time optimizations such as quantization, KV-cache management, continuous batching, speculative decoding and parallelism strategies.
• Develop and tune GPU kernels (CUDA, Triton) for attention, GEMM and MoE operators on the latest NVIDIA architectures.
• Build benchmarking and load-testing frameworks to measure end-to-end serving performance under realistic workloads and SLOs.
• Partner with infrastructure teams to deploy and scale inference on Kubernetes-based GPU clusters, including autoscaling, fault tolerance and observability.
• Keep up with inference research and bring new techniques into production.

Required profile

• 3+ years of hands-on experience optimizing deep learning inference on NVIDIA GPUs, preferably for large language models.
• Bachelor's or higher degree in Computer Science, Engineering or a related field.
• Experience with at least one mainstream inference serving engine (vLLM, SGLang, TensorRT-LLM, Triton Inference Server or equivalent).

Required skills

• Strong knowledge of LLM inference internals: attention mechanisms, KV cache, batching strategies, quantization and parallelism.
• Proficiency in CUDA and/or Triton kernel programming, with a solid understanding of modern GPU architecture.
• Strong software engineering fundamentals in Python and C++.
• Nice to have: open-source inference projects, low-precision inference (FP8, INT4, FP4), GPU profiling tools (Nsight Systems, PyTorch Profiler), Kubernetes deployment experience, SLO-driven capacity planning, and deep learning training framework experience (PyTorch, DeepSpeed, Megatron-LM, FSDP).