Platform Engineer, RL & Eval Data

Home > Python programming jobs

Platform Engineer, RL & Eval Data in Remote new

Anyone AI
Full time
Email

Remote
11/14/2026
Report

Argentina

Responsibilities

Maintain and improve evaluation, RL, and annotation platforms, including bug fixes, technical-debt reduction, testing, and documentation.
Design and ship features across databases, backend services, APIs, and internal tools.
Scale distributed systems for larger workloads with predictable performance and cost.
Build RL environments and the infrastructure for sandboxes, environment images, rollouts, and reward computation.
Operate reliable and secure production systems through monitoring, alerting, incident response, access control, untrusted-code isolation, and client-data protection.
Build tooling for task authors, reviewers, and project leads and support rapid-turnaround data samples.
Collect high-throughput rollouts with parallel episodes, timeouts, resource limits, logs, and trajectories.
Build evaluation harnesses, verifiers, graders, and reward functions for LLMs and LLM agents.
Support contributor and reviewer workflows, quality control, calibration, and scaled human-data processes.

Requirements

At least 5 years of experience in software, data, or ML engineering, with recent work in AI/ML data, model evaluation, annotation, or developer tooling.
Strong backend engineering experience in Python, including clean, tested, and maintainable code.
Experience designing APIs, data models, relational database schemas, migrations, and query-performance improvements using Postgres or similar systems.
Enough TypeScript and React experience to build and maintain internal tools.
Experience designing distributed systems with long-running jobs, orchestration, workflow engines, queues, retries, idempotency, and failure recovery.
Experience with containers, container orchestration, and at least one major cloud provider.
Experience with observability, including logging, metrics, tracing, and alerting, plus security fundamentals such as secrets management, isolation, and access control.
Hands-on experience building RL environments for LLM agents, operating isolated sandboxes at scale, building environment images, collecting rollouts, and feeding rewards into RL post-training pipelines.
Hands-on experience building or running LLM or LLM-agent evaluation harnesses or benchmarks and understanding evaluation concepts such as pass rates, pass@k, headroom, discrimination, and reward hacking.
Experience with annotation, labeling, or human-data tooling and with scaling manual processes into measured workflows.
A track record of running and maintaining production systems, fluent English, strong written communication, independent execution, and comfort working in ambiguity.
Preferred qualifications include contributions to open-source evaluation or RL frameworks and experience building tools for a large distributed contributor workforce.
Daily careful use of AI coding agents and rigorous review of their output.

Benefits

100% remote work arrangement.
Opportunity to work on frontier AI training and evaluation data platforms.
Core ownership and broad technical scope on a small, fast-moving team.

Anyone AI

Anyone AI empowers businesses to harness the full potential of artificial intelligence with intuitive, scalable solutions that drive innovation and efficiency. By blending cutting‑edge technology with user‑friendly design, we help organizations transform data into actionable insights and streamli...

Similar positions

Research Engineer, Synthetic Data

Pragmatike
Full time

Remote
11/14/2026

APAC, US

Sr. Data Engineer

KnowBe4
Full time

Others
11/14/2026

Bengaluru, India

Snr. Data Engineer

KnowBe4
Full time

Others
11/14/2026

Cape Town, South Africa

Platform Engineer (Backend Infrastructure)

Focal Systems
Full time

Remote
11/14/2026
Salary: $102k - $118k/yr

Portugal

Software Engineer I, Data Platforms

Abnormal AI
Full time

Remote
11/14/2026
Salary: $122k-$176k

United States