Senior PySpark Developer
Sunil KumarUnited StatesSENIORPosted 5d ago
PySparkPythonApache SparkSpark SQLAzure DatabricksDelta LakeAzure Data LakeAzure Data FactoryETL/ELTData EngineeringCI/CDAzure DevOpsGit
About the Role
We are actively looking for a highly experienced PySpark Developer for a long-term contract opportunity with TCS supporting a leading Healthcare client.
Key Responsibilities
• Design and support enterprise-scale data pipelines
• Optimize data pipelines for performance and quality
• Work across both cloud and on-premises environments
Requirements
• 9+ years of Data Engineering experience
• 6+ years of hands-on PySpark development
• Strong experience with Azure Databricks, SparkSQL, Python, and ETL/ELT development
• Expertise in Delta Lake, Azure Data Lake Storage Gen2, and Lakehouse Architecture
• Experience working in both Cloud and On-Premises environments
• Strong understanding of data pipeline optimization, CDC, incremental loads, and data quality frameworks
• Azure Data Factory experience preferred