Senior Software Engineer II

Home > Python programming jobs

Senior Software Engineer II in Others

Nielsen
Full time
Email

Others
09/25/2026
Report

Mumbai, India

Responsibilities

Own the end-to-end architecture, strategy, and operational maturity of the Databricks Lakehouse platform.
Design workspace topology, cluster policies, compute strategy, multi-cloud deployment patterns, and platform reliability practices.
Define and enforce Unity Catalog governance, including catalogs, schemas, access controls, lineage, audit logging, and metadata standards.
Establish data engineering standards for Delta Lake, Delta Live Tables, medallion architecture, Z-ordering, liquid clustering, partitioning, and file compaction.
Industrialize platform provisioning and pipeline deployment through Terraform or Databricks Asset Bundles, CI/CD tooling, and monitoring.
Own Databricks cost transparency and optimization through right-sizing, workload isolation, chargeback/showback, and budget alerting.
Architect AI-native capabilities such as feature stores, vector search, and GenAI-ready pipelines within governed platform boundaries.
Govern the rollout of Databricks Genie, AI/BI Dashboards, Mosaic AI, and other native AI use cases.
Partner with engineering, analytics, data science, CloudOps, DevOps, and business stakeholders to deliver self-service platform capabilities.
Lead a Databricks community of practice through standards, enablement sessions, mentorship, and delivery support.

Requirements

8–10+ years of experience in data engineering, data platform architecture, or distributed systems, including 3+ years architecting and operating Databricks environments at scale.
Direct ownership and hands-on experience building, managing, running, and scaling a production-grade, large-scale, multi-tenant Databricks Data Platform.
Expertise with Unity Catalog, Delta Lake, Delta Live Tables, Databricks Workflows, Databricks SQL, and cluster and compute policy design.
Experience architecting AI-native platform capabilities, including feature stores, vector search or indexes, and GenAI-ready pipelines.
Experience using AI-assisted and agentic engineering practices, such as coding copilots and automated testing or operations agents.
Deep hands-on expertise with Apache Spark batch and structured streaming for large-scale processing and performance tuning.
Expert-level proficiency in Python and SQL, with working knowledge of dbt.
Experience with Terraform or Databricks Asset Bundles and CI/CD tooling for infrastructure and pipeline automation.
Strong understanding of data security, RBAC, ABAC, PII handling, and compliance requirements in governed lakehouse environments.
Ability to evaluate architectural trade-offs, identify technical or cost risks, and communicate decisions clearly to technical and non-technical stakeholders.
Preferred qualifications include Databricks Certified Data Engineer Professional or Databricks Certified Platform Architect certification.
Ability to use a Bottom Line Up Front communication style and explain the rationale behind platform decisions.

Nielsen

Nielsen provides audience measurement and analytics that quantify who watches, listens, and buys across TV, digital, radio, and streaming. It sells ratings, cross-platform datasets, and advertising effectiveness tools to media owners, advertisers, and agencies on subscription and consulting agree...

Similar positions

Senior DevOps Engineer

Lemon.io
Full time

Remote
10/30/2026

Europe, APAC

Software Engineer (Python)

Commvault
Full time

Others
10/30/2026

Pune, India

Software Engineer, Platform, SRE

Motive
Full time

Others
10/30/2026

Bengaluru, India

Senior Software Engineer, Core Platform

Reddit
Full time

USA
10/30/2026
Salary: $191k-$267k

Remote

Senior Software Engineer - Agent Experiences

Alpaca
Full time

Remote
10/30/2026
Salary: Competitive

AMER