PySpark Developer
Sreeja TirunagaruRemoteMIDPosted 2h ago
PySparkPythonApache SparkJavaKafkaSQL
About the Role
We are looking for a PySpark Developer to join our team. This role focuses on designing and maintaining scalable data processing pipelines.
Key Responsibilities
• Design, develop, and maintain scalable data processing pipelines using PySpark and Python
• Build and optimize ETL/ELT workflows for large-scale structured and unstructured datasets
• Develop distributed data processing applications using Apache Spark
• Write clean, reusable, and performance-optimized code using Python and Java
• Work extensively with Spark SQL, DataFrames, Datasets, and RDDs
• Perform Spark performance tuning, including partitioning, caching, and shuffling
• Ensure data quality, reliability, and scalability across data engineering solutions
Requirements
• Strong hands-on experience in PySpark, Python, and Apache Spark
• Good understanding of Java programming
• Experience in large-scale data processing and distributed computing
• Knowledge of Kafka and SQL
• Strong understanding of ETL/ELT pipelines and data engineering best practices