Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Senior Inference Engineer

Home > Python programming jobs

Senior Inference Engineer in USA

  • vCluster Labs
  • Full time
  • Email
  • Remote

Responsibilities

  • Deploy LLMs to production across one or more machines on GPU infrastructure and own the full query-to-response pipeline.
  • Build and operate model-serving infrastructure using vLLM, SGLang, or TensorRT-LLM.
  • Optimize inference latency and cost using quantization, batching, caching, and routing.
  • Develop production-grade inference infrastructure in Python or Golang.
  • Build the initial inference platform with the CTO, lead its roadmap, and partner with Product on future development.

Requirements

  • Production experience deploying and serving LLMs with vLLM, SGLang, or TensorRT-LLM.
  • Hands-on experience with inference optimization techniques including quantization, batching, caching, and routing.
  • Strong production programming experience in Python or Golang.
  • Strong communication skills for explaining technical concepts to technical and non-technical stakeholders.
  • Familiarity with Docker and Kubernetes is a bonus.
  • Generative AI experience with PyTorch or Transformers is a bonus.
  • Understanding of CUDA, NCCL, GPU drivers, and related libraries is a bonus.
  • Knowledge of model architectures and fine-tuning approaches is a bonus.
  • Experience with NVIDIA Dynamo is a bonus.

Benefits

  • Competitive compensation package including equity.
  • Health, dental, vision, and life insurance for employees and eligible dependents, with benefits varying by country.
  • Flexible working schedule.
  • Remote-first workplace flexibility with a distributed global team.

Salary: $190k - $220k/yr

vCluster Labs

vCluster Labs builds open-source and commercial tools that simplify Kubernetes multi-tenancy and platform operations for platform engineering and SRE teams. Its products include vCluster (virtual clusters for tenant isolation), DevPod, JsPolicy, and an Infrastructure Tenancy Platform for AI with ...

Similar positions

Software Engineer I, Backend

  • Affirm
  • Full time
  • Canada
  • 11/21/2026
  • Remote

Machine Learning Training Framework Engineer

  • ByteDance
  • Full time
  • USA
  • 11/21/2026
  • San Jose, CA

Senior Full Stack Engineer

  • Pavago
  • Full time
  • Remote
  • 11/21/2026
  • Jamaica

Lead Cloud Infrastructure Engineer (GCP)

  • Egen
  • Full time
  • Others
  • 11/21/2026
  • Hyderābād, India

Lead DevOps Engineer

  • L3Harris
  • Full time
  • USA
  • 11/21/2026
  • Salt Lake City, UT