Senior Software Engineer, Spark Platform

Home > SCALA programming jobs

Senior Software Engineer, Spark Platform in USA new

DoorDash
Full time
Email

USA
10/14/2026
Report

Seattle, WA

Responsibilities

Set multi-year technical direction for DoorDash’s in-house Spark-on-Kubernetes platform.
Own distributed-systems problems involving runtime performance, shuffle architecture, multi-tenant scheduling, and large-scale failure modes.
Shape architecture across the Spark runtime, remote shuffle service, cluster scheduler, and reliability tooling.
Partner with the Engineering Manager on technical roadmap, hiring, interview design, and team shape.
Mentor engineers through design reviews and technical guidance.
Represent Spark Platform in cross-team architecture forums and guide how data, analytics, and ML workloads use the platform.

Requirements

Bachelor’s, master’s, or PhD in Computer Science or equivalent.
6+ years of industry experience designing and operating distributed systems at scale.
Deep hands-on Apache Spark experience covering internals, query execution, shuffle, and the executor/driver model at platform scale.
Experience operating Spark on Amazon EMR, Databricks, or an in-house deployment, with emphasis on platform operations rather than authoring individual Spark jobs.
Production experience with remote or external shuffle systems, batch or big-data schedulers, or observability and SRE patterns for distributed compute platforms.
Strong production Kubernetes experience, including operator patterns, executor pod lifecycle, network topology, and multi-tenant failure modes.
Familiarity with data lake table formats such as Apache Iceberg or Delta Lake and with query and SQL engines that read them.
Technical leadership experience on a platform team, including setting direction, mentoring, and partnering with management on roadmap and hiring.
Professional experience with Scala, Java, Python, or Go, plus strong SQL.
Located in or willing to relocate to the Bay Area, Seattle, or New York City.

Benefits

Hybrid work arrangement requiring residence in San Francisco, Sunnyvale, Seattle, or New York City.
401(k) plan with employer matching.
16 weeks of paid parental leave.
Medical, dental, and vision benefits.
Wellness benefits and wellness expense reimbursement.
Commuter benefits match.
Flexible paid time off or vacation and paid sick time for salaried roles.
11 paid holidays.
Disability and basic life insurance.
Family-forming assistance and a mental health program.
Equity grant opportunities in addition to base salary.

Salary: $160k - $235k/yr

DoorDash

At DoorDash, our mission to empower local economies shapes how our team members move quickly and always learn and reiterate to support merchants, Dashers and the communities we serve. We are a technology and logistics company that started with door-to-door delivery, and we are looking for team me...

Similar positions

Back End Staff Software Engineer

Agoda
Full time

Others
10/15/2026

Bangkok, Thailand

Senior Staff Engineer

Netradyne
Full time

Others
10/14/2026

Bengaluru, India

Senior Scientific Software Developer [635]

D-Wave Systems
Full time

Canada
10/14/2026
Salary: Competitive

Burnaby

Senior Staff Machine Learning Engineer

Airbnb
Full time

USA
10/13/2026
Salary: $244k-$305k

Remote

Senior Software Engineer (App-Framework)

Databricks
Full time

Others
10/12/2026

Bengaluru, India