Jobs → Engineering → PySpark Developer

About the Role We are looking for an experienced PySpark Developer for a cloud transformation and data modernization initiative. Key Responsibilities • Design and implement data pipelines using PySpark and Apache Spark • Work with Cloudera Data Platform (CDP) for data processing • Utilize Apache Iceberg and Data Lakehouse architecture • Develop workflows with Apache Airflow • Apply SQL, ETL/ELT, and data modeling skills • Leverage AWS services including S3, Glue, Lambda, and EMR • Focus on data pipeline development, performance tuning, and optimization • Implement Git, CI/CD, and DevOps practices Requirements • 5–7 years of experience in PySpark and Apache Spark • Experience with Cloudera Data Platform (CDP) • Knowledge of Apache Iceberg and Data Lakehouse architecture • Experience with Apache Airflow and workflow orchestration • Strong SQL, ETL/ELT, and data modeling skills • Exposure to AWS services, including S3, Glue, Lambda, and EMR • Understanding of data pipeline development, performance tuning, and optimization • Familiarity with Git, CI/CD, and DevOps practices