Software Engineer, Data Infra

Home > Python programming jobs

Software Engineer, Data Infra in USA

Thinking Machines Lab
Full time
Email

USA
10/02/2026
Report

San Francisco, CA

Responsibilities

Design, build, and operate scalable, fault-tolerant infrastructure for distributed LLM research workloads across compute, orchestration, and multimodal storage.
Develop high-throughput systems for data ingestion, processing, transformation, training-data catalogs, deduplication, quality checks, and search.
Build traceability, reproducibility, and quality-control systems across the data lifecycle.
Implement and maintain monitoring and alerting for platform reliability and performance.
Collaborate with research teams to improve data quality, accelerate training cycles, and deliver new capabilities.
Own projects end-to-end and contribute across different technology stacks and teams.

Requirements

Bachelor’s degree or equivalent experience in computer science, engineering, or a similar field.
Proficiency in at least one backend language, specifically Python or Rust.
Fluency with distributed compute frameworks such as Apache Spark or Ray.
Deep familiarity with cloud infrastructure, data lake architectures, and batch and streaming pipelines.
Preferred hands-on experience with Kafka, dbt, Terraform, and Airflow.
Preferred experience building a web crawler.
Preferred extensive experience scaling deduplication, data mining, and search systems.
Strong knowledge of file formats and storage systems such as Parquet and Delta Lake.
Ability to work collaboratively with cross-functional partners, take initiative, and own projects end-to-end.

Benefits

The role is based in San Francisco, California.
Visa sponsorship is available, with the company committed to working through the visa process for the right fit, though success is not guaranteed for every candidate or role.
Generous health, dental, and vision benefits.
Unlimited paid time off.
Paid parental leave.
Relocation support as needed.

Salary: $300k - $400k/yr

Thinking Machines Lab

Thinking Machines Lab pioneers advanced AI research, blending cutting‑edge theory with practical applications to solve complex problems across industries. Our interdisciplinary team pushes the boundaries of machine learning, neural architecture, and cognitive modeling to deliver innovative soluti...

Similar positions

Senior Software Engineer - Machine Learning

Anduril Industries
Full time

USA
10/25/2026
Salary: $190k - $252k/yr

Broomfield, CO

2027 Software Engineer

Point72
Full time

Others
10/25/2026

Bengaluru, India

Staff Software Engineer | AI

LawPay
Full time

Remote
10/25/2026
Salary: $150k-$190k

United States

Senior Mission Software Engineer

Varda Space Industries
Full time

USA
10/25/2026
Salary: $169k - $216k/yr

El Segundo, CA

Software Engineer, AI Integrations

Zyte
Full time

Remote
10/25/2026
Salary: Competitive

Poland