Your browser does not support javascript! Please enable it, otherwise web will not work for you.

ML Infra Engineer, Compute

Home > Javascript/Typescript programming jobs

ML Infra Engineer, Compute in USA

  • Physical Intelligence
  • Full time
  • Email
  • San Francisco, CA

Responsibilities

  • Design and build intelligent multi-tenant scheduling and placement systems based on hardware, topology, availability, cost, and priority.
  • Implement quota management, priority tiers, preemption, queueing, fairness, and resource-sharing policies.
  • Build multi-cluster control-plane capabilities for job lifecycle management, migration, failover, and rescheduling across GPU and TPU clusters.
  • Monitor and optimize accelerator utilization, capacity, cost efficiency, and large-scale training performance.
  • Implement fault detection, automatic recovery, health checking, node management, and scaling for thousands of accelerators.
  • Extend scheduling and orchestration to inference workloads and model deployment on physical robots.
  • Build dashboards, alerting, SLOs, and debugging tools that improve observability and researcher experience.

Requirements

  • Strong software engineering fundamentals.
  • Experience building or operating job scheduling or resource management systems at scale.
  • Experience with large-scale GPU and/or TPU compute clusters.
  • Familiarity with SLURM, Kubernetes, GKE, K3S, or comparable orchestration systems.
  • Ability to reason about resource allocation, bin-packing, priority scheduling, and multi-tenancy.
  • Understanding of long-running, multi-node, straggler-sensitive, and topology-dependent ML training workloads.
  • Ownership of systems end-to-end from design through operation and collaboration with research teams.
  • Bonus: experience with federated scheduling, TPU infrastructure, Borg, YARN, Mesos, Linux systems engineering, networking, infrastructure-as-code, NCCL, collective communication, topology-aware placement, capacity planning, cloud cost optimization, JAX, or PyTorch.

Physical Intelligence

Physical Intelligence builds general-purpose AI for the physical world, developing foundation models and full-stack robotic systems for robots and other actuated devices. The privately held company was founded in 2024 and focuses on taking designs from prototype to high-volume production, includi...

Similar positions

Product Engineer, Frontend Focus

  • Clera
  • Full time
  • Others
  • 11/20/2026
  • Munich, Germany

Staff Performance Engineer, JavaScript Engine

  • Mozilla
  • Full time
  • Remote
  • 11/20/2026
  • Salary: EUR 55,650 - 74,200
  • Belgium, Finland

Senior Engineer, AS

  • D.B. Group
  • Full time
  • Others
  • 11/20/2026
  • Pune, India

Senior Software Engineer, Scientific AI

  • Seqera
  • Full time
  • Others
  • 11/20/2026
  • Barcelona, Spain

Senior Software Engineer, Cloud Engineering

  • Nextdoor
  • Full time
  • USA
  • 11/19/2026
  • Salary: $187k - $238k/yr
  • Remote