Databricks Developer
Lucy Deepika VelaidumUnited StatesSENIORPosted 14m ago
DatabricksAWSPySparkSparkAWS S3IAMAWS GlueDelta LakeUnity CatalogDelta Live TablesDatabricks WorkflowsCI/CDData Engineering
About the Role
We are looking for an experienced Databricks Developer to join our team. This role focuses on building, optimizing, and managing large-scale data processing solutions using Databricks and AWS services.
Key Responsibilities
• Design, develop, and maintain scalable data engineering solutions using Databricks
• Build and optimize large-scale data pipelines using PySpark and Apache Spark
• Integrate Databricks with AWS services, including S3, IAM, and Glue
• Implement Delta Lake architecture and data governance solutions
• Work with Unity Catalog, Delta Live Tables, and Databricks Workflows
• Optimize Spark jobs for performance and scalability
• Implement CI/CD pipelines and data workflow automation
• Provide technical guidance and support to development teams
Requirements
• Senior-level hands-on experience in Databricks development
• Strong experience with Databricks on AWS
• Hands-on experience with AWS S3, IAM, and Glue integration
• Must currently reside within commuting distance of Malvern, PA
• Must be available to work onsite Tuesday, Wednesday, and Thursday from Day 1
• Strong PySpark and Apache Spark performance tuning experience
• Experience with Delta Lake
• Knowledge of Unity Catalog and data governance
• Experience with Delta Live Tables (DLT)
• Databricks Workflows and job orchestration
• CI/CD implementation for data engineering pipelines
• Databricks Certified Data Engineer Professional certification
• Financial services or investment industry experience