Staff Engineer, Test Automation

Home > Python programming jobs

Staff Engineer, Test Automation in USA

Shield AI
Full time
Email

USA
10/22/2026
Report

San Mateo, CA

Responsibilities

Own the technical strategy, architecture, and roadmap for automated testing, verification, and MLOps quality across the Hivemind ecosystem.
Design and maintain scalable test frameworks for autonomy software, backend services, APIs, operator-facing applications, and distributed hardware environments.
Lead functional, integration, regression, system, performance, reliability, and end-to-end testing across simulation, edge-compute, software-in-the-loop, and hardware-in-the-loop environments.
Build automated ML validation pipelines covering data quality, training reproducibility, model accuracy, robustness, regression, latency, resource utilization, and system integration.
Establish versioning and traceability for datasets, models, configurations, evaluation results, and deployment artifacts through continuous training workflows.
Develop scenario-based validation for autonomy models, including degraded sensing or communications, distribution shifts, edge cases, and representative mission conditions.
Create observability, analytics, and failure-triage capabilities for software behavior, model and data drift, inference health, test results, and production performance.
Build Python automation, test harnesses, simulators, stubs, mocks, and synthetic-data capabilities to improve test execution, coverage, reporting, and developer productivity.
Collaborate with software, autonomy, machine learning, data, simulation, and systems engineers to define verification strategies and improve designs.
Develop and govern AI-assisted engineering workflows using coding agents and LLM-based tools for test generation, log analysis, debugging, and failure triage.

Requirements

Typically 8+ years of relevant experience in software engineering, test infrastructure, developer tooling, MLOps, systems integration, or systems verification, or an equivalent combination of experience and demonstrated impact.
At least 5 years of experience building scalable automation frameworks or developer tooling in Python.
Demonstrated success designing test, CI/CD, or MLOps infrastructure used across multiple engineering teams.
Experience validating machine learning systems across data, training, evaluation, packaging, deployment, and monitoring lifecycles.
Experience with ML quality risks, model-performance baselines, automated evaluation suites, release thresholds, and candidate-to-production comparison workflows.
Experience testing GPU-accelerated workloads and multi-tenant Kubernetes environments, including scheduling, allocation, utilization, resource contention, RBAC, quotas, isolation, and scheduling behavior.
Experience with performance benchmarking, profiling, observability, integrated hardware and software qualification, distributed systems, backend services, APIs, and regression strategies.
Strong asynchronous and concurrent Python programming skills for scalable automation and parallel test execution.
Experience with package and dependency management, reproducible environments, build systems, automated observability, log collection, analytics, reporting, root-cause analysis, and Linux-based development environments.
Preferred qualifications include experience with MLflow, Kubeflow, Weights & Biases, SageMaker, Vertex AI, GPU orchestration and profiling tools, NVIDIA GPU infrastructure, autonomy or robotics models, embedded or edge-compute platforms, production hardware qualification, fault-injection and recovery testing, containers, Kubernetes, cloud infrastructure, infrastructure as code, Go or TypeScript, C or C++ interoperability, and safety-critical verification standards such as DO-178C and MIL-STD-882.

Benefits

Full-time employees receive pay within the listed range plus bonus, benefits, and equity.
Temporary employees receive pay within the listed range plus a temporary benefits package applicable after 60 days of employment.
Offers are contingent on a cleared background and possible reference check.
Military fellows and part-time employees are not eligible for benefits.
Shield AI provides equal employment opportunity and disability or special-needs accommodation.

Salary: $188k - $282k/yr

Shield AI

Shield AI builds autonomous systems for military and national security customers, combining its Hivemind autonomy software with V-BAT and X-BAT unmanned aircraft and Aechelon simulation technologies. The privately held company sells hardware, software, and related services to U.S. and allied defe...

Similar positions

Staff Security Engineer

Quince
Full time

Others
10/26/2026

Bengaluru, India

Test Automation Lead

Accenture
Full time

Canada
10/25/2026

Mississauga

Staff AI Engineer

Typeform
Full time

UK
10/25/2026

Remote

Staff Software Engineer | AI

LawPay
Full time

Remote
10/25/2026
Salary: $150k-$190k

United States

Staff Machine Learning Engineer

Quora
Full time

Remote
10/25/2026
Salary: $220k-$321k

USA, Canada