Backend Inference Framework Engineer Graduate (AML Inference) - 2027 Start

About the role

Data AML is ByteDance's machine learning mid-platform, providing training and inference systems for recommendation and advertising for businesses such as Douyin, Toutiao and Xigua Video. It supplies machine learning computing power to internal business units and researches general and innovative algorithms. ByteDance is recruiting graduates to join the team in 2027; successful candidates must be able to commit to an onboarding date by the end of 2027.

Key responsibilities

• Design and implement the architecture of model inference services, building a high-performance, highly available and scalable enterprise-level inference system for large-parameter, high-complexity AI models.
• R&D and optimization of core inference framework modules such as inference engine scheduling, monitoring and alerting, and canary release, improving performance and resolving bottlenecks in high-concurrency, large-model scenarios.
• Track the latest inference technologies, select and innovate them against business needs, and promote upgrades and standardization of the team's technical system.

Required profile

• Completing or recently completed a Bachelor's or Master's degree in computing or a related discipline.
• Experience in R&D projects of high-concurrency distributed services, with familiarity with service latency and resource optimization.
• Good learning and execution abilities, problem analysis and abstraction skills, and strong cross-team collaboration, communication, presentation and document writing skills.
• Preferred: practical project experience and source code understanding of high-concurrency services or frameworks such as Redis, RocksDB, BRPC or gRPC.
• Preferred: understanding of GPU operation and experience optimizing GPU service resource management and control.

Required skills

• Familiarity with basic Linux commands.
• Solid C/C++ programming skills and knowledge of data structures and algorithms.
• Understanding of multi-threaded concurrency, including threads, synchronization locks and thread pools, and basic performance tuning.