Databricks Platform Engineering Lead
Kotesh K.United StatesLEADPosted 6h ago
DatabricksAzure DatabricksUnity CatalogApache SparkPySparkPythonSQLDelta LakeLakehouse architectureAzure Data Lake StorageAzure Data FactoryTerraformCI/CDInfrastructure as code
About the Role
We are looking for a Databricks Platform Engineering Lead to join our team at AllianceIT. This senior leadership role focuses on managing and optimizing Databricks platforms and cloud data infrastructure.
Key Responsibilities
• Lead platform engineering initiatives with Databricks and cloud data platforms
• Manage workspace administration, security, and governance using Unity Catalog
• Oversee infrastructure provisioning and CI/CD practices
• Direct team of platform engineers and data engineers
• Ensure optimal performance, cost efficiency, and reliability of data platforms
Requirements
• 10+ years of overall technology experience
• 5+ years of hands-on experience with Databricks, cloud data platforms, or platform engineering
• 3+ years of experience in a Platform Lead, Databricks Lead, or similar role
• Strong hands-on experience with Azure Databricks, Unity Catalog, Apache Spark, Python, and SQL
• Expertise in Delta Lake, Lakehouse architecture, and Azure Data Lake Storage
• Experience with Terraform for infrastructure as code
• Understanding of CI/CD and automated deployment practices
• Leadership experience with globally distributed teams