Own and improve the GPU serving stack built on vLLM and SGLang, including configuration, scheduling, and framework changes.
Accelerate large-scale evaluation and synthetic data generation and design the inference pipelines supporting those workloads.
Select, profile, benchmark, and tune the inference engine used by the API service, adapting the API layer as needed.
Integrate Liquid Foundation Models into vLLM, SGLang, and Hugging Face Transformers and maintain the public integrations.
Diagnose throughput and latency problems with researchers and serving engineers and carry performance improvements from profiling through benchmarked results.
Requirements
Hands-on experience running LLM inference on GPUs with a serving framework such as vLLM or SGLang beyond default configuration.
First-principles understanding of transformer forward passes, autoregressive GPU decoding, and latency, throughput, and memory trade-offs.
Solid understanding of GPU fundamentals relevant to inference.
Experience profiling and benchmarking inference workloads and applying the resulting measurements.
Strong Python and PyTorch skills and comfort reading and modifying large open-source codebases.
Merged contributions to vLLM, SGLang, or FlashInfer; lower-level CUDA or Triton optimization experience; and high-throughput batch inference experience are helpful but not required.
Benefits
Full-time role available in San Francisco, Boston, or remotely; the listed location type is hybrid.
Competitive base salary plus equity; final offers reflect skills, experience, and location.
Unlimited PTO and company-wide Refill Days and long weekends.
Comprehensive health coverage, with details varying by location.
Paid leave for pregnancy, parenting, medical needs, and caring for family.
Salary: Competitive
Liquid AI
We build efficient general-purpose AI at every scale.