Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Python with Spark Developer-Chennai

Home > Python programming jobs

Python with Spark Developer-Chennai in Others

  • Chennai, India

Responsibilities

  • Design, develop, and maintain robust Python, PySpark, and SQL ingestion, transformation, and enrichment pipelines.
  • Write and optimize complex SQL queries, analytical UDFs, stored procedures, and window functions for aggregation and reporting.
  • Collaborate with data architects, data scientists, and business analysts to translate requirements into technical specifications.
  • Perform unit testing, integration testing, code reviews, production support, and bug fixing.
  • Maintain CI/CD pipelines using Git, Jenkins, and Docker for automated build, testing, and deployment.
  • Monitor production workloads and troubleshoot performance bottlenecks, memory issues, and job failures.
  • Document data lineage, pipeline designs, and operational runbooks in Confluence and SharePoint.
  • Mentor junior engineers and promote Python, Spark optimization, and data-engineering best practices.
  • Evaluate emerging technologies, contribute to AI/ML innovation, and deliver proof-of-concepts for CEFS.
  • Participate in Agile ceremonies and backlog grooming.

Requirements

  • 5–7 years of experience as a Python Developer working with Spark or PySpark.
  • Strong hands-on experience with Python, PySpark, advanced SQL, and ETL in a data engineering environment.
  • Experience with PySpark DataFrames, Spark SQL, Structured Streaming, partitioning, caching, broadcast joins, and performance tuning.
  • Knowledge of Python packages including NumPy and pandas for data cleaning, wrangling, analysis, and visualization.
  • Experience with MS-SQL or Oracle, Linux/Unix environments, testing phases, documentation, and production support.
  • Experience with build tools such as Maven and DevOps tools such as Bitbucket and Jenkins.
  • Knowledge of object-oriented programming, APIs, design patterns, development principles, and multi-integration applications.
  • Knowledge of front-end technologies, preferably Flask, is desirable.
  • Bachelor’s degree or equivalent.
  • Ability to collaborate, simplify complex topics, deliver results, maintain attention to detail, and learn diverse technologies.

Benefits

  • Innovation-focused work culture with ongoing learning opportunities.
  • Flat, non-hierarchical structure with opportunities to work directly with senior partners and clients.
  • Diverse, inclusive, and meritocratic culture.
  • Work location: Chennai.

Capco

Capco, a Wipro company, is a global technology and management consultancy specializing in driving digital transformation in the financial services industry. With a growing client portfolio comprising of over 100 global organizations, Capco operates at the intersection of business and technology b...

Similar positions

Python with Django

  • Capco
  • Full time
  • Others
  • 09/12/2026
  • Chennai, India

Software Engineer (Python/Django)

  • Data Meaning
  • Full time
  • Remote
  • 09/12/2026
  • Brazil, Latin America

Software Engineer (Python, React)

  • 66degrees
  • Full time
  • Others
  • 09/11/2026
  • Pune, India

SDET - Python, Go

  • Nutanix
  • Full time
  • Others
  • 09/11/2026
  • Bengaluru, India

Full Stack Developer (Python, React)

  • 66degrees
  • Full time
  • Others
  • 09/11/2026
  • Pune, India

Listings

by


Job Type

Language

Country

CodingJobBoard.com © 2026