Loading...
Mid LevelFull-timeCI/CDData
Salary Range
$20,400 - $27,600 /year
This salary is estimated based on similar roles. The actual salary may vary.
Job Title: Data Engineer (PySpark / Databricks)
Experience: 5–9 Years Location: Pune (Hybrid – Capco Office)
Job Summary
We are looking for a skilled Data Engineer with strong expertise in PySpark, Databricks, and modern data engineering practices. The ideal candidate will have hands-on experience in building scalable data pipelines, working with large datasets, and leveraging cloud-based data platforms.
Key Responsibilities Design, develop, and maintain scalable ETL/ELT data pipelines Work extensively with PySpark and Apache Spark for large-scale data processing Build and manage workflows using Apache Airflow Develop and optimize data solutions on Databricks (Jobs, Delta Lake) Work with cloud-based data lakes (S3 or equivalent) Write efficient and complex SQL queries for data transformation and analysis Run and manage Spark workloads on EMR Serverless or other managed Spark platforms Ensure data quality, reliability, and performance optimization of pipelines Must Have Skills Strong hands-on experience with PySpark and Apache Spark internals Experience with Databricks (Jobs, Delta Lake) Proficiency in Apache Airflow for workflow orchestration Solid experience building ETL/ELT pipelines at scale Strong SQL skills and experience with Data Warehouse (DWH) systems Experience running Spark workloads on EMR Serverless or managed Spark platforms Hands-on experience with cloud data lakes (S3 or equivalent) Good to Have Skills Experience with Delta Lake / Apache Iceberg Exposure to streaming frameworks (Spark Structured Streaming, Kafka) Familiarity with CI/CD pipelines for data engineering workflows Knowledge of data governance, cataloging, and lineage tools
Free to sign up. Takes 30 seconds.
About Capco
Capco is a global consulting firm specializing in the financial services industry, providing strategic advisory, technology implementation, and transformation services. Headquartered in the United Kingdom, the company has established a strong presence across major financial centers worldwide, offering expertise to help institutions navigate complex regulatory landscapes, leverage digital technologies, and drive operational efficiency. With a history rooted in addressing the unique challenges of the financial sector, Capco has built a reputation for delivering tailored solutions that align with the evolving needs of its clients. Capco’s core offerings include digital transformation, regulatory compliance, data analytics, and cloud computing services, designed to empower financial institutions to adapt to market changes and regulatory requirements. The firm’s solutions are characterized by their integration of cutting-edge technology with deep industry knowledge, enabling clients to enhance decision-making, improve customer experiences, and optimize risk management. Capco primarily serves banks, asset managers, insurance companies, and other financial organizations, focusing on delivering measurable outcomes that drive long-term value and competitive advantage. Notable for its client-centric approach and collaborative problem-solving, Capco has been instrumental in helping numerous financial institutions achieve their strategic objectives. The company’s market position is strengthened by its ability to combine technical expertise with a deep understanding of the financial industry’s nuances. By fostering innovation and leveraging emerging technologies, Capco continues to position itself as a trusted partner for organizations seeking to thrive in an increasingly digital and regulated environment.
