Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Software Engineer, Evaluation Platform / Infra

Home > Python programming jobs

Software Engineer, Evaluation Platform / Infra in USA

  • Thinking Machines Lab
  • Full time
  • Email
  • San Francisco, CA

Responsibilities

  • Design, build, and maintain a platform for authoring, running, tracking, and analyzing model evaluations.
  • Develop evaluation libraries, distributed backend systems, data pipelines, APIs, and user-facing applications end to end.
  • Build abstractions for evaluation tasks, environments, graders, datasets, and model outputs.
  • Improve reproducibility and trust through versioning, provenance, observability, failure recovery, and quality controls.
  • Partner with researchers to convert bespoke workflows and bottlenecks into self-serve systems used across teams.
  • Collaborate with the Research Tooling team and other pre-training, post-training, and applied teams.

Requirements

  • Bachelor’s degree or equivalent practical experience in computer science, engineering, machine learning, or a related field.
  • At least two years of post-graduate software engineering or ML engineering experience, excluding internships.
  • Hands-on experience building or maintaining evaluations, benchmarks, graders, or model-quality systems for large language or multimodal models.
  • Strong software engineering fundamentals and experience building reliable, maintainable systems.
  • Proficiency in at least one backend programming language, with Python and Rust used primarily.
  • Experience with databases, data pipelines, distributed systems, or other data-intensive infrastructure.
  • Ability to work across the stack and own projects from problem discovery through deployment and operation.
  • Experience collaborating with cross-functional partners and subject-matter experts.
  • Preferred experience includes building frameworks, SDKs, developer tools, distributed job execution, workflow orchestration, sandboxed environments, large-scale data processing, polished data-inspection interfaces, model evaluation, research collaboration, or end-to-end work on small teams.

Benefits

  • Annual salary range of $300,000–$475,000 USD.
  • Health, dental, and vision benefits.
  • Unlimited paid time off.
  • Paid parental leave.
  • Relocation support as needed.
  • Visa sponsorship is available.
  • The role is based in San Francisco, California.

Salary: $300k - $475k/yr

Thinking Machines Lab

Thinking Machines Lab pioneers advanced AI research, blending cutting‑edge theory with practical applications to solve complex problems across industries. Our interdisciplinary team pushes the boundaries of machine learning, neural architecture, and cognitive modeling to deliver innovative soluti...

Similar positions

Software Engineer in Test

  • Roadie
  • Full time
  • USA
  • 09/13/2026
  • Salary: Competitive
  • Remote

Principal Software Engineer - Builder Experience

  • Elastic
  • Full time
  • Remote
  • 09/13/2026
  • Salary: €73k-€116k
  • Greece

Principal Software Engineer - Builder Experience

  • Elastic
  • Full time
  • Remote
  • 09/13/2026
  • Salary: zł 369k-zł 584k
  • Poland

Senior Core Infrastructure Engineer

  • Oracle
  • Full time
  • USA
  • 09/13/2026
  • Salary: Competitive
  • Nashville, TN

Senior Software Engineer, ML Ops

  • PathAI
  • Full time
  • USA
  • 09/13/2026
  • Salary: $128k - $196k/yr
  • Boston, MA

Listings

by


Job Type

Language

Country

CodingJobBoard.com © 2026