Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Member of Technical Staff

Home > Python programming jobs

Member of Technical Staff in USA

  • San Jose, CA

Responsibilities

  • Build and own end-to-end evaluation suites for LLM capabilities, including task specifications, grading logic, and researcher dashboards.
  • Operate evaluation harnesses as production services with sandboxed execution, tool calling, multi-turn and agentic tasks, deterministic replay, and caching.
  • Own data pipelines for collection, filtering, deduplication, labeling, synthetic data generation, and quality gates.
  • Build and maintain RL training environments, including reward functions, simulators, verifiers, and task generators.
  • Collaborate with researchers to turn hypotheses into runnable evaluations and environments and interpret results.
  • Measure evaluation regressions, environment health, throughput, and cost while identifying misleading metric changes.
  • Improve research-code quality through testing, versioned datasets and evaluations, and clear training, inference, and evaluation boundaries.

Requirements

  • At least 3 years of software engineering experience, including 1–2 years of hands-on LLM training, evaluation, or data-infrastructure experience.
  • Deep Python fluency and practical distributed-systems experience across unreliable workers and machine fleets.
  • Experience building evaluation infrastructure, including tasks, graders, and supporting systems.
  • Working knowledge of supervised fine-tuning and reinforcement learning for language models.
  • Production-oriented engineering practices, including testing and reproducibility.
  • Ability to work effectively with researchers and turn ambiguous ideas into measurable systems.
  • Preferred qualifications include agentic evaluation, tool use, code execution, sandbox environments, multi-turn tasks, training frameworks, inference stacks, data-labeling operations, human-preference collection, open-source evaluation frameworks, public benchmarks, verifiers, and simulation environments.

Benefits

  • Full-time position with a US base salary range of $180,000–$450,000 annually.
  • Additional compensation components and benefits may be included depending on the specific role and shared if an employment offer is extended.

Salary: $180k - $450k/yr

Hark

Hark is a forward‑thinking technology firm that empowers businesses to harness the power of data-driven insights and automation. By blending cutting‑edge analytics with intuitive user experiences, we help organizations streamline operations, unlock new growth opportunities, and stay ahead in a ra...

Similar positions

Technical Consultant

  • Esri
  • Full time
  • USA
  • 11/21/2026
  • Salary: $84k - $142k/yr
  • Redlands, CA

Senior Software Engineer

  • G-P
  • Full time
  • Remote
  • 11/21/2026
  • India

Senior Software Engineer

  • AMD
  • Full time
  • USA
  • 11/21/2026
  • Salary: $204k - $306k/yr
  • San Jose, CA

Software Engineer II, Backend

  • Affirm
  • Full time
  • Remote
  • 11/21/2026
  • Salary: PLN 259,000 - 379,000
  • Poland

Senior Software Engineer

  • 3Y Health
  • Full time
  • Remote
  • 11/21/2026
  • Argentina