Python Spark Databricks Developer
Aadarsh D.United StatesMIDPosted 14d ago
PythonPySparkApache SparkDatabricksETLELTUnixLinuxShell ScriptingControl-MSpark Performance TuningData IntegrationData TestingData Deployment
About the Role
We are looking for an experienced Data Engineer/Developer with strong hands-on experience in data processing and ETL/ELT pipelines.
Key Responsibilities
• Design and implement data processing solutions using Python, PySpark, and Apache Spark
• Develop and maintain ETL/ELT pipelines for large-scale data
• Utilize Databricks for data engineering tasks
• Implement Unix/Linux Shell Scripting for automation
• Configure Control-M for job scheduling and workflow orchestration
• Perform Spark performance tuning and optimization
• Provide production support and troubleshooting
• Ensure data integration, testing, and deployment
Requirements
• Strong hands-on experience with Python, PySpark, and Apache Spark
• Experience with Databricks platform
• Proficiency in ETL/ELT and large-scale data pipelines
• Knowledge of Unix/Linux Shell Scripting
• Experience with Control-M for job scheduling
• Understanding of Spark performance tuning
• Experience with data integration, testing, and deployment
Benefits
• Healthcare / Health Insurance