Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Senior Data Engineer

Home > Python programming jobs

Senior Data Engineer in Others

  • Illumina
  • Full time
  • Email
  • Bengaluru, India

Responsibilities

  • Own workspace- and account-level Databricks architecture, including Unity Catalog, compute policies, Lakeflow Jobs, identity, secrets, audit logging, storage integration, upgrades, and environment promotion.
  • Design, build, and review infrastructure-as-code, CI/CD, observability, reusable standards, reference architectures, and self-service platform patterns.
  • Own platform reliability, including availability, capacity, incident management, root-cause analysis, disaster recovery/business continuity, and service-level objectives.
  • Manage Databricks FinOps activities including consumption monitoring, cost optimization, budgets, anomaly management, and chargeback/showback.
  • Implement platform security and governance controls covering identity federation, SCIM, service principals, network controls, secrets, RBAC/ABAC, masking, PII, lineage, and audit.
  • Enable and operate Databricks capabilities for AI/ML workloads, including MLflow, Model Serving, vector search, and GenAI infrastructure patterns.
  • Build and maintain PySpark, SQL, Delta Lake, and Auto Loader reference implementations and paved-road patterns.
  • Own the platform roadmap, contribute to architecture decisions and engineering standards, lead code reviews, and mentor engineers.

Requirements

  • 3–5 years of professional data platform or data engineering experience, including at least 2 years administering and operating Databricks in production as an enterprise platform.
  • Experience designing, building, and reviewing infrastructure-as-code using Terraform or an equivalent, plus CI/CD for data platforms using GitHub Actions, AWS CodePipeline, or similar tools.
  • Strong Python and PySpark skills and advanced SQL knowledge.
  • Experience with monitoring, alerting, incident management, runbooks, capacity management, and performance management.
  • Experience with Databricks cost management and FinOps practices.
  • Experience with platform security and compliance, including identity, service principals, secrets, network controls, audit logging, RBAC, PII handling, and controls such as SOX.
  • Understanding of distributed systems and system design for large-scale data processing.
  • Strong AWS fundamentals, including S3, IAM, VPC, networking, and KMS, and knowledge of Databricks integrations with AWS.
  • Demonstrated use of AI-assisted engineering tools, or willingness and ability to adopt them, across coding, infrastructure-as-code, testing, documentation, and troubleshooting.
  • Strong written and verbal communication skills with the ability to work with global stakeholders and lead technical discussions.
  • Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent demonstrable experience.
  • Preferred qualifications include Databricks Certified Data Engineer Professional or AWS Certified Solutions Architect certifications, compliance-driven platform experience, mentoring experience, platform migration experience, and global delivery experience.

Benefits

  • Inclusive workplace with employee resource groups, career development opportunities, cultural awareness initiatives, and social responsibility opportunities.
  • Equal opportunity employer with accommodation support available during the application or interview process.
  • This role is not eligible for visa sponsorship.

Illumina

Illumina builds next-generation DNA sequencing systems, reagents/consumables, and bioinformatics software used by research, clinical, pharmaceutical, and agricultural genomics labs. Its business model combines sale of instruments with recurring consumables, sequencing services, and cloud analysis...

Similar positions

Backend Engineer (Europe)

  • Kalepa
  • Full time
  • Remote
  • 10/26/2026
  • Salary: $80k-$105k
  • Europe

Sr. AI/ML Engineer II

  • MetroStar
  • Full time
  • USA
  • 10/26/2026
  • Salary: $200k - $240k/yr
  • Reston, VA

Staff Security Engineer

  • Quince
  • Full time
  • Others
  • 10/26/2026
  • Bengaluru, India

ChatGPT Performance Engineer

  • OpenAI
  • Full time
  • USA
  • 10/26/2026
  • Salary: $325k-$405k
  • Remote

Junior Backend Engineer (Europe)

  • Kalepa
  • Full time
  • Remote
  • 10/25/2026
  • Salary: $60k-$70k
  • Europe

Listings

by


Job Type

Language

Country

CodingJobBoard.com © 2026