Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Senior Software Engineer, GPU Infrastructure

Home > Python programming jobs

Senior Software Engineer, GPU Infrastructure in Canada

  • Toronto

Responsibilities

  • Build, deploy, and operate Kubernetes-based GPU/TPU superclusters across multiple clouds.
  • Optimize infrastructure for AI/ML training performance, reliability, and cost using RDMA, NCCL, and high-speed interconnects.
  • Diagnose and resolve infrastructure bottlenecks, performance degradation, and system failures.
  • Create self-service interfaces and workflows for researchers to monitor, debug, and optimize training jobs.
  • Translate emerging needs involving JAX, PyTorch, and distributed training into scalable infrastructure solutions.
  • Promote observability, automation, infrastructure-as-code, maintainability, and resilience.
  • Collaborate with AI researchers, ML engineers, cloud providers, and other engineering teams.
  • Contribute to code reviews, documentation, mentorship, and knowledge transfer.
  • Participate in a compensated 24x7 on-call rotation.

Requirements

  • Deep experience with ML/HPC infrastructure, GPU/TPU clusters, distributed training frameworks, and high-performance computing environments.
  • Proven ability to deploy, manage, and troubleshoot Kubernetes clusters at scale for AI workloads.
  • Proficiency in Python for ML tooling and Go for systems engineering.
  • Familiarity with Linux internals, RDMA networking, and performance optimization for ML workloads.
  • Experience collaborating with AI researchers or ML engineers on infrastructure challenges.
  • Ability to identify bottlenecks, propose solutions, and drive impact in a fast-paced environment.
  • Open-source contribution experience is preferred.

Benefits

  • Weekly lunch stipend of $75/£75 or local equivalent.
  • Full health and dental benefits with a separate mental health budget.
  • RRSP matching, 401K, and pension scheme options.
  • 100% parental leave top-up for up to six months for either parent.
  • Annual arts and culture, fitness and wellness, quality time, workspace improvement, education, and learning benefits.
  • Six weeks of paid vacation, equivalent to 30 working days.
  • Travel budget for remote employees to visit other offices and an annual company offsite.
  • Remote-friendly work arrangement with offices in Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin, and Seoul.
  • Coworking benefit for employees not near an office and a $500 home office stipend.
  • Office employees receive a daily lunch program, snacks, and community and social events.

Cohere

Cohere is the leading security-first enterprise AI company. We build cutting-edge foundation models and end-to-end AI products designed to solve real-world business problems. We partner closely with companies to deliver seamless integration, full customization, and easy-to-use solutions for their...

Similar positions

Software Engineer in Test

  • Roadie
  • Full time
  • USA
  • 09/13/2026
  • Salary: Competitive
  • Remote

Principal Software Engineer - Builder Experience

  • Elastic
  • Full time
  • Remote
  • 09/13/2026
  • Salary: €73k-€116k
  • Greece

Principal Software Engineer - Builder Experience

  • Elastic
  • Full time
  • Remote
  • 09/13/2026
  • Salary: zł 369k-zł 584k
  • Poland

Senior Devops Engineer

  • Woliba
  • Full time
  • USA
  • 09/13/2026
  • Remote

Senior Core Infrastructure Engineer

  • Oracle
  • Full time
  • USA
  • 09/13/2026
  • Salary: Competitive
  • Nashville, TN

Listings

by


Job Type

Language

Country

CodingJobBoard.com © 2026