Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Senior Inference Engineer

Home > Python programming jobs

Senior Inference Engineer in USA new

  • vCluster Labs
  • Full time
  • Email
  • Remote

Responsibilities

  • Deploy LLMs to production across one or more machines on GPU infrastructure and own the full query-to-response pipeline.
  • Build and operate model-serving infrastructure using vLLM, SGLang, or TensorRT-LLM.
  • Optimize inference latency and cost using quantization, batching, caching, and routing.
  • Develop production-grade inference infrastructure in Python or Golang.
  • Build the initial inference platform with the CTO, lead its roadmap, and partner with Product on future development.

Requirements

  • Production experience deploying and serving LLMs with vLLM, SGLang, or TensorRT-LLM.
  • Hands-on experience with inference optimization techniques including quantization, batching, caching, and routing.
  • Strong production programming experience in Python or Golang.
  • Strong communication skills for explaining technical concepts to technical and non-technical stakeholders.
  • Familiarity with Docker and Kubernetes is a bonus.
  • Generative AI experience with PyTorch or Transformers is a bonus.
  • Understanding of CUDA, NCCL, GPU drivers, and related libraries is a bonus.
  • Knowledge of model architectures and fine-tuning approaches is a bonus.
  • Experience with NVIDIA Dynamo is a bonus.

Benefits

  • Competitive compensation package including equity.
  • Health, dental, vision, and life insurance for employees and eligible dependents, with benefits varying by country.
  • Flexible working schedule.
  • Remote-first workplace flexibility with a distributed global team.

Salary: $190k - $220k/yr

vCluster Labs

vCluster Labs builds open-source and commercial tools that simplify Kubernetes multi-tenancy and platform operations for platform engineering and SRE teams. Its products include vCluster (virtual clusters for tenant isolation), DevPod, JsPolicy, and an Infrastructure Tenancy Platform for AI with ...

Similar positions

Software Engineer in Test

  • Roadie
  • Full time
  • USA
  • 09/13/2026
  • Salary: Competitive
  • Remote

Principal Software Engineer - Builder Experience

  • Elastic
  • Full time
  • Remote
  • 09/13/2026
  • Salary: €73k-€116k
  • Greece

Principal Software Engineer - Builder Experience

  • Elastic
  • Full time
  • Remote
  • 09/13/2026
  • Salary: zł 369k-zł 584k
  • Poland

Senior Devops Engineer

  • Woliba
  • Full time
  • USA
  • 09/13/2026
  • Remote

Senior Core Infrastructure Engineer

  • Oracle
  • Full time
  • USA
  • 09/13/2026
  • Salary: Competitive
  • Nashville, TN

Listings

by


Job Type

Language

Country

CodingJobBoard.com © 2026