Member of Technical Staff - Evaluations

Home > Other programming jobs

Member of Technical Staff - Evaluations in USA new

Liquid AI
Full time
Email

USA
11/11/2026
Report

San Francisco, CA

Responsibilities

Own the internal evaluation suite and infrastructure end to end, keeping it reliable, optimized, reproducible, and scalable.
Design held-out evaluations for capabilities including agentic workflows, computer use, and cybersecurity, with documented construction and contamination checks.
Build evaluations targeted at small edge models and debug unexpected scores by reviewing individual generated outputs.
Convert customer and open-source feedback into environments for data, training, and evaluation, and partner with other teams on their evaluation needs.
Release open-source evaluations, write papers, and publish failure-mode reviews of popular evaluations.
Drive post-training decisions through internal evaluations and help co-own the evaluation function.

Requirements

Strong evaluation-design ability, including translating high-level capabilities into accurate, general-purpose, hard-to-game evaluations that distinguish model quality.
Prior experience building evaluations that demonstrate coverage, reproducibility, contamination handling, and useful signal, including at least one benchmark, evaluation paper, or substantial contribution built from scratch.
Experience with evaluation fundamentals including contamination, saturation, judge bias, and variance.
Hands-on experience running and managing evaluations within an evaluation, training, engineering, or DevOps team.
Strong software engineering skills sufficient to own evaluation infrastructure and debug inference and infrastructure failures at scale independently.
Helpful qualifications include model training experience, publications or research experience, deep systems expertise, and familiarity with small edge models.

Benefits

Full-time hybrid role based in San Francisco.
Competitive base salary plus equity; final offers reflect skills, experience, and location.
Unlimited paid time off and company-wide Refill Days and long weekends.
Comprehensive health coverage, with details varying by location.
Paid leave for pregnancy, parenting, medical needs, and caring for family.
Real compute for running evaluations at scale and the opportunity to open-source and publish evaluation work.

Salary: Competitive

Liquid AI

We build efficient general-purpose AI at every scale.

Similar positions

Technology Engineer - Principal - ITS - 1044

City and County of San Francisco
Full time

Remote
11/11/2026
Salary: $171k-$215k

United States

Staff Software Engineer, Mobile (Android)

Google
Full time

USA
11/11/2026
Salary: $207k - $300k/yr

Mountain View, CA

Staff+ Software Engineer, Claude Design

Anthropic
Full time

USA
11/11/2026
Salary: $405k - $485k/yr

San Francisco, CA

Staff Software Engineer, Onboarding

Reddit
Full time

USA
11/11/2026
Salary: $217k-$303k

Remote

Solutions Architect - Search Specialist

Elastic
Full time

UK
11/11/2026
Salary: Competitive

Remote

Listings

by

Job Type

Language

Country