Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Senior Software Engineer, GPU Infrastructure

Home > Python programming jobs

Senior Software Engineer, GPU Infrastructure in Canada

  • Toronto

Responsibilities

  • Build, deploy, and operate Kubernetes-based GPU/TPU superclusters across multiple clouds.
  • Optimize infrastructure for AI/ML training performance, reliability, and cost using RDMA, NCCL, and high-speed interconnects.
  • Diagnose and resolve infrastructure bottlenecks, performance degradation, and system failures.
  • Create self-service interfaces and workflows for researchers to monitor, debug, and optimize training jobs.
  • Translate emerging needs involving JAX, PyTorch, and distributed training into scalable infrastructure solutions.
  • Promote observability, automation, infrastructure-as-code, maintainability, and resilience.
  • Collaborate with AI researchers, ML engineers, cloud providers, and other engineering teams.
  • Contribute to code reviews, documentation, mentorship, and knowledge transfer.
  • Participate in a compensated 24x7 on-call rotation.

Requirements

  • Deep experience with ML/HPC infrastructure, GPU/TPU clusters, distributed training frameworks, and high-performance computing environments.
  • Proven ability to deploy, manage, and troubleshoot Kubernetes clusters at scale for AI workloads.
  • Proficiency in Python for ML tooling and Go for systems engineering.
  • Familiarity with Linux internals, RDMA networking, and performance optimization for ML workloads.
  • Experience collaborating with AI researchers or ML engineers on infrastructure challenges.
  • Ability to identify bottlenecks, propose solutions, and drive impact in a fast-paced environment.
  • Open-source contribution experience is preferred.

Benefits

  • Weekly lunch stipend of $75/£75 or local equivalent.
  • Full health and dental benefits with a separate mental health budget.
  • RRSP matching, 401K, and pension scheme options.
  • 100% parental leave top-up for up to six months for either parent.
  • Annual arts and culture, fitness and wellness, quality time, workspace improvement, education, and learning benefits.
  • Six weeks of paid vacation, equivalent to 30 working days.
  • Travel budget for remote employees to visit other offices and an annual company offsite.
  • Remote-friendly work arrangement with offices in Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin, and Seoul.
  • Coworking benefit for employees not near an office and a $500 home office stipend.
  • Office employees receive a daily lunch program, snacks, and community and social events.

Cohere

Cohere is the leading security-first enterprise AI company. We build cutting-edge foundation models and end-to-end AI products designed to solve real-world business problems. We partner closely with companies to deliver seamless integration, full customization, and easy-to-use solutions for their...

Similar positions

Graduate Software Engineer

  • Synack
  • Full time
  • UK
  • 10/04/2026
  • Remote

Software Engineer - Cloud Images

  • Canonical
  • Full time
  • Remote
  • 10/04/2026
  • EMEA, Americas

Senior Backend Engineer - Client integrations

  • Kalepa
  • Full time
  • Remote
  • 10/04/2026
  • Salary: $80k-$165k
  • Europe

Senior Software Engineer

  • Muon Space
  • Full time
  • USA
  • 10/04/2026
  • Salary: $192k - $213k/yr
  • Denver, CO

Software Engineer III

  • JPMorgan Chase
  • Full time
  • USA
  • 10/04/2026
  • Salary: Competitive
  • Columbus, OH

Listings

by


Job Type

Language

Country

CodingJobBoard.com © 2026