Senior Site Reliability Engineer

Home > Python programming jobs

Senior Site Reliability Engineer in USA

ByteDance
Full time
Email

USA
10/20/2026
Report

San Jose, CA

Responsibilities

Own the stability, performance optimization, and issue resolution of large-scale generative AI training and inference systems.
Design and improve resource scheduling systems, including utilization, prioritization, fine-grained scheduling, and scheduling explainability.
Plan, deploy, operate, and optimize high-speed RDMA networks and heterogeneous GPU, NPU, and XPU clusters.
Own the architecture, selection, operation, performance, reliability, and stability of high-performance storage systems.
Build capacity planning methods and deliver reliable, timely, and cost-effective resource expansion and configuration optimization.
Develop unified compute pools for co-located training and inference workloads and establish operating methods for compute efficiency and cost control.
Build monitoring, alerting, and automated remediation across the training and inference path and resolve complex production incidents.
Partner with algorithm, product, and operations teams on MLOps solutions and establish engineering practices for large-scale generative AI systems.

Requirements

Bachelor's degree or above in Computer Science, Software Engineering, or a related field.
3+ years of experience in large-scale machine learning, generative AI MLOps, or closely related infrastructure work.
Proficiency in Python and/or Go, strong Linux development and operations skills, and expert shell scripting and automation capabilities.
Expertise with Docker or Kata containerization and Kubernetes orchestration, including hands-on operation and optimization of large-scale heterogeneous GPU or NPU clusters.
Knowledge and practical optimization experience with RDMA high-speed networking and distributed or object storage.
Expertise designing and optimizing resource scheduling systems and improving scheduling policies and utilization.
Experience maintaining and optimizing large-scale generative AI training or inference systems and resolving issues across the full training and inference path.
Experience with co-located training and inference compute pools or offline inference efficiency, plus strong communication, collaboration, ownership, and production operations skills.
Preferred qualifications include productionizing large-scale generative AI models; managing heterogeneous clusters and capacity planning in multi-cloud or hybrid-cloud environments; tuning distributed training and inference systems; optimizing RDMA or storage performance; delivering compute capacity and operating frameworks with measurable cost or utilization improvements; designing distributed systems; and running fault-injection or chaos exercises.

ByteDance

ByteDance is a global incubator of platforms at the cutting edge of commerce, content, entertainment and enterprise services - over 2.5bn people interact with ByteDance products including TikTok. Creation is the core of ByteDance's purpose. Our products are built to help imaginations thrive. This...

Similar positions

Backend Engineer (Europe)

Kalepa
Full time

Remote
10/26/2026
Salary: $80k-$105k

Europe

Sr. AI/ML Engineer II

MetroStar
Full time

USA
10/26/2026
Salary: $200k - $240k/yr

Reston, VA

Staff Security Engineer

Quince
Full time

Others
10/26/2026

Bengaluru, India

ChatGPT Performance Engineer

OpenAI
Full time

USA
10/26/2026
Salary: $325k-$405k

Remote

Junior Backend Engineer (Europe)

Kalepa
Full time

Remote
10/25/2026
Salary: $60k-$70k

Europe