Databricks Engineer
Sai SirishUnited StatesSENIORPosted 2h ago
Data engineeringDatabricksAWSS3IAMGlueLambdaEC2KinesisRedshiftCloudWatchPythonPySparkSQLApache Spark
About the Role
We are looking for a Databricks Engineer to join our team. This role requires extensive experience in data engineering with a focus on Databricks and AWS services.
Key Responsibilities
• Develop and maintain data engineering solutions using Databricks
• Work with core AWS services including S3, IAM, Glue, Lambda, EC2, Kinesis, Redshift, and CloudWatch
• Utilize Python (PySpark) and SQL for data processing and analysis
• Optimize Apache Spark performance through partitioning, shuffles, caching, and join optimization
Requirements
• 10 years of experience in Data engineering
• At least 2 years of hands-on Databricks experience
• Strong proficiency in Python (PySpark) and SQL
• Solid understanding of Apache Spark internals
• Experience with AWS services