Inference Systems Backend Engineer

Home > C/C++ programming jobs

Inference Systems Backend Engineer in Others

ByteDance
Full time
Email

Others
09/07/2026
Report

Singapore, Singapore

Responsibilities

Develop and performance-optimize Volcano Engine large model training and inference systems.
Optimize model computation, tune thousand-GPU training clusters, build distributed LLM inference systems, and schedule large-scale inference traffic.
Solve high-concurrency, high-reliability, and high-scalability challenges for training and inference traffic at hundreds-of-billions-of-tokens scale.
Research and introduce architectures involving subgraph matching, compiler optimization, and model quantization.
Integrate GPUs, NPUs, and TPUs with training and inference frameworks.
Improve utilization across globally distributed GPU clusters through elastic scheduling, GPU oversubscription, and task orchestration.
Collaborate with algorithm teams to jointly optimize algorithms and systems.

Requirements

Proficiency in C/C++ and Python development in Linux environments.
Experience with large-scale machine learning systems or search, advertising, and recommendation systems.
Familiarity with at least one machine learning framework, such as TensorFlow, PyTorch, MXNet, or an in-house framework.
Familiarity with at least one large model training or inference framework, including vLLM, TensorRT-LLM, SGLang, or Megatron-LM.
Strong independent problem-solving, teamwork, communication, learning, and self-motivation skills.
Preferred: experience designing large-scale distributed-system architectures.
Preferred: understanding of GPU hardware architecture and software stacks such as CUDA and cuDNN, with GPU performance analysis and optimization experience.
Preferred: Master’s or PhD-level research background in distributed systems, parallel computing, programming languages and compilers, networking, or storage systems.

ByteDance

ByteDance is a global incubator of platforms at the cutting edge of commerce, content, entertainment and enterprise services - over 2.5bn people interact with ByteDance products including TikTok. Creation is the core of ByteDance's purpose. Our products are built to help imaginations thrive. This...

Similar positions

Embedded Development Engineer

Emerson Electric Co.
Full time

Others
10/04/2026

Pune, India

Intermediate Embedded Software Engineer

MITRE
Full time

USA
10/04/2026
Salary: $104k - $155k/yr

New Bedford, MA

Senior Embedded Software Engineer

Apex Technology, Inc.
Full time

USA
10/04/2026
Salary: $185k - $210k/yr

Los Angeles, CA

Bluetooth Development Engineer

Visteon Corporation
Full time

Others
10/04/2026

Chennai, India

Software Engineer II

Microsoft
Full time

USA
10/04/2026
Salary: $102k - $219k/yr

Redmond, WA