Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Senior Staff Inference Performance Engineer

Home > Python programming jobs

Senior Staff Inference Performance Engineer in USA new

  • d-Matrix
  • Full time
  • Email
  • Remote

Responsibilities

  • Diagnose performance problems across orchestration, scheduling, placement, networking, autoscaling, inference internals, runtimes, host operating systems, and accelerators.
  • Isolate root causes to specific system layers and owners, fixing issues directly or handing them off with reproducible findings and evidence.
  • Protect throughput, latency, and stability targets by identifying gaps between orchestrated workloads and bare-metal performance.
  • Assess fleet-scale deployment concerns including routing, placement, and shared-resource contention.
  • Capture CPU placement, memory, NUMA locality, and network-path requirements in orchestration configuration.
  • Build lightweight telemetry and benchmarks for prototype stacks.
  • Convert recurring investigations into playbooks and mentor engineers in a profiling-first performance methodology.

Requirements

  • 8+ years of hands-on performance or systems engineering experience, including resolving cross-layer performance regressions.
  • Demonstrated ability to profile unfamiliar, sparsely documented systems and identify performance overhead.
  • Deep knowledge of Linux internals, NUMA, CPU pinning and cpusets, PCIe, memory and cache behavior, and network paths.
  • Experience running performance-sensitive workloads on Kubernetes and addressing overhead from scheduler defaults, network overlays, and orchestration abstractions.
  • Working knowledge of LLM inference concepts including batching, KV-cache, prefill/decode, quantization, and speculative decoding.
  • Proficiency in Python and C/C++ for instrumentation, debugging, and cross-stack patching.
  • Strong technical writing for hardware, firmware, and platform teams, plus sound judgment about ownership and handoffs.
  • Fluency with AI and agentic tools for investigation, code search, log triage, and instrumentation drafting, with the ability to verify results against real data.
  • Preferred experience with vLLM, SGLang, TensorRT-LLM, speculative decoding, disaggregated or heterogeneous serving, KV-cache management, request and batch scheduling, weight prefetch and residency, GPU programming and profiling, pre-production hardware, or Rust.
  • Startup experience or comfort working in an environment with shifting priorities.

Benefits

  • Location is Santa Clara, California or remote within the United States.
  • The company describes an inclusive, collaborative, and equal-opportunity workplace.

Salary: $175k - $265k/yr

d-Matrix

d-Matrix builds AI inference computing platforms for data centers, combining custom silicon with systems, networking, and software. Its flagship Corsair platform and JetStream fabric focus on low-latency, energy-efficient generative AI inference at scale. Founded in 2019 and headquartered in Sant...

Similar positions

iOS Engineer - App Analysis

  • Data Theorem
  • Full time
  • Canada
  • 10/16/2026
  • Remote

DevOps Engineer - AI

  • IFS
  • Full time
  • Remote
  • 10/16/2026
  • APAC

IT Engineer

  • ElevenLabs
  • Full time
  • Remote
  • 10/16/2026
  • Worldwide

Senior Threat Intelligence Engineer

  • Coalition, Inc.
  • Full time
  • Canada
  • 10/16/2026
  • Salary: C$107k-C$190k
  • Remote

Client Platform Engineer

  • OpenAI
  • Full time
  • USA
  • 10/16/2026
  • Salary: $347k - $445k/yr
  • San Francisco, CA

Listings

by


Job Type

Language

Country

CodingJobBoard.com © 2026