Production Engineer

Home > Python programming jobs

Production Engineer in Others

ByteDance
Full time
Email

Others
09/09/2026
Report

Singapore, Singapore

Responsibilities

Manage production operations and stability for AML training, inference, and storage systems across scheduling, orchestration, Kubernetes/GPU clusters, distributed training, online inference, and Parameter Server/NoSQL storage.
Build and maintain SLO/SLA frameworks, observability, alerting, on-call processes, incident diagnosis, self-healing, disaster recovery, and post-incident review practices.
Develop engineering capabilities including canary and gradual deployments, automated rollback, health inspections, pre-flight checks, capacity forecasting, and elastic scaling.
Lead resource governance and optimization across GPU, CPU, storage, and network infrastructure, including quota management, cost attribution, and performance tuning.

Requirements

Bachelor's degree or above in Computer Science, Software Engineering, Artificial Intelligence, or a related field.
Proficiency in Linux and at least one of Shell, Python, Go, or C++.
Familiarity with machine learning training and inference architectures, Kubernetes, GPU clusters, or distributed storage systems.
Experience with production issue troubleshooting, performance analysis, and automation platform development.
Strong ownership, analytical and problem-solving skills, and the ability to drive cross-functional collaboration.
Preferred experience with large-scale training, inference, or storage platforms, SLO governance, FinOps practices, NoSQL systems, or open-source infrastructure projects.
Preferred hands-on experience operating and governing large-scale Kubernetes containerized clusters, including Operators, declarative operations, and release protection mechanisms.
Preferred familiarity with recommendation and advertising system architectures and AI infrastructure components such as Parameter Servers or KV Caches, including Mooncake.

ByteDance

ByteDance is a global incubator of platforms at the cutting edge of commerce, content, entertainment and enterprise services - over 2.5bn people interact with ByteDance products including TikTok. Creation is the core of ByteDance's purpose. Our products are built to help imaginations thrive. This...

Similar positions

Backend Engineer (Europe)

Kalepa
Full time

Remote
10/26/2026
Salary: $80k-$105k

Europe

Sr. AI/ML Engineer II

MetroStar
Full time

USA
10/26/2026
Salary: $200k - $240k/yr

Reston, VA

Staff Security Engineer

Quince
Full time

Others
10/26/2026

Bengaluru, India

ChatGPT Performance Engineer

OpenAI
Full time

USA
10/26/2026
Salary: $325k-$405k

Remote

Junior Backend Engineer (Europe)

Kalepa
Full time

Remote
10/25/2026
Salary: $60k-$70k

Europe