Your browser does not support javascript! Please enable it, otherwise web will not work for you.

LLM Platform Engineer

Home > Python programming jobs

LLM Platform Engineer in USA

  • AllSpice
  • Full time
  • Email
  • Boston, MA

Responsibilities

  • Partner with engineers and product managers to identify workflow bottlenecks, prioritize problems, and own solutions from requirements and architecture through rollout and iteration.
  • Own evaluation and observability systems for AI products, including automated and human review, regression detection, and comparisons across models, prompts, and pipelines.
  • Build internal applications and self-service tools for inspecting AI results, investigating failures, and collecting feedback, with potential customer-facing extensions.
  • Build scripts, data pipelines, integrations, and automations connecting internal systems, third-party services, LLMs, and Model Context Protocol tools.
  • Create unified analytics layers from evaluation results, customer feedback, and telemetry while maintaining data quality and schema evolution.
  • Design RAG-based information repositories and tooling for component data and domain knowledge.
  • Integrate AI insights with CI/CD workflows and help improve AI workflow reliability, cost, latency, and cache effectiveness.
  • Document technical findings, communicate with technical and non-technical audiences, mentor teammates, and champion code quality, testing, and reliability.

Requirements

  • 4+ years of software engineering experience focused on tooling, data platforms, or full-stack web development, with senior-level ownership of dependable systems or products.
  • Strong SQL skills and experience designing data pipelines and storage schemas, managing data quality and schema evolution, and using analytics tools.
  • Proficiency in Python scripting and automation for data pipelines, integrations, and internal tools.
  • Experience building web applications or customer-facing tools with modern frameworks such as Vue, React, or TypeScript.
  • Practical experience shipping or maintaining LLM-enabled systems, including evaluation pipelines, observability, or integrations, with awareness of non-determinism and cost, latency, quality, and reliability tradeoffs.
  • Bachelor’s degree or higher in a technology-related field, or equivalent practical experience.
  • Strong communication, customer focus, project management, stakeholder collaboration, autonomy, and comfort with ambiguity.
  • Preferred familiarity with business analytics and data visualization tools, LLM evaluation frameworks, GitHub Actions, monitoring and observability tools, and common web architectures and APIs.

Benefits

  • Flexible work with preferred availability for hybrid work from the San Francisco or Boston flex offices 2–3 times per week.
  • Competitive salary and equity, health, dental, and vision benefits, generous PTO, and a home office stipend.
  • Supportive colleagues, flexible work, and the opportunity to make a significant impact.

Salary: $160k - $190k/yr

AllSpice

AllSpice is the collaboration platform for modern hardware teams. We bring software-style workflows to electronics development, so schematics, reviews, and decisions are easy to share, track, and scale. DRCY is our AI design review agent. It runs automated, auditable first-pass checks directly in...

Similar positions

Software Development Engineer

  • Ford Motor
  • Full time
  • USA
  • 11/21/2026
  • Salary: $85k - $143k/yr
  • Cary, NC

Software Engineer II, Backend

  • Affirm
  • Full time
  • Remote
  • 11/21/2026
  • Salary: GBP 101,000 - 149,000
  • UK

Senior Engineer, Security Automation

  • Twilio
  • Full time
  • USA
  • 11/21/2026
  • Salary: $145k - $213k/yr
  • Remote

Principal Security Engineer, Detection & Response

  • Circle
  • Full time
  • Remote
  • 11/21/2026
  • Salary: €110k-€170k
  • Ireland, USA

Lead Data Engineer

  • Appian
  • Full time
  • USA
  • 11/21/2026
  • Salary: $150k - $275k/yr
  • McLean, VA