PySpark Developer
Navin IRemoteSENIORPosted 7h ago
PySparkApache SparkCloudera Data PlatformApache IcebergData LakeLakehouse architectureApache AirflowAWS Data Engineering servicesSQLETLELTdata pipeline developmentData modelingperformance tuningGitCI/CDDevOps
About the Role
We are looking for an experienced PySpark Developer to join a large-scale cloud transformation and data modernization initiative. This role focuses on building scalable data pipelines and developing modern cloud-based data platforms.
Key Responsibilities
• Design and implement data pipelines using PySpark and Apache Spark
• Work with Cloudera Data Platform (CDP) and Data Lake / Lakehouse architecture
• Develop workflows using Apache Airflow
• Utilize AWS Data Engineering services
• Perform data modeling, performance tuning, and optimization
Requirements
• Strong hands-on experience with PySpark and Apache Spark
• Experience with Cloudera Data Platform (CDP)
• Knowledge of Apache Iceberg and Data Lake / Lakehouse architecture
• Proficiency with Apache Airflow for workflow orchestration
• Strong SQL, ETL/ELT, and data pipeline development skills
• Experience with Git, CI/CD, and DevOps practices