Databricks Platform Engineering Lead
Amrinder SinghUnited StatesSENIORPosted 46m ago
Azure DatabricksDatabricks AdministrationCloud ArchitectureData EngineeringApache SparkPySparkPythonSQLDelta LakeLakehouse ArchitectureAzure Data Lake StorageAzure Data FactoryDatabricks WorkflowsJob OrchestrationETL/ELT PipelinesTerraformCI/CD AutomationDatabricks Workspace ProvisioningEnvironment ManagementCluster ConfigurationCompute ManagementCost OptimizationPlatform SecurityIdentity & Access ManagementUnity CatalogData GovernancePermissionsData LineageAzure NetworkingPrivate EndpointsSecure ConnectivityPlatform MonitoringLoggingPerformance OptimizationProduction SupportArchitecture ReviewsSolution DesignCode ReviewsProduction ReadinessAzure DevOpsMicrosoft PurviewAzure MonitorMLflow
About the Role
We are looking for an experienced Databricks Platform Engineering Lead to lead enterprise-level Databricks platform engineering, administration, governance, and technical delivery for a global reinsurance organization.
Key Responsibilities
• Lead end-to-end Databricks platform engineering and technical delivery
• Provide technical oversight to Data Engineering teams
• Drive architecture reviews, solution design, code reviews, and production readiness
• Manage globally distributed engineering, DevOps, QA, and production-support teams
• Coordinate releases, deployments, SIT/UAT, and production stabilization
• Drive stakeholder communication, risk management, and delivery governance
Requirements
• Strong hands-on experience with Azure Databricks Platform Engineering and Administration
• Expertise in Databricks Account & Workspace Administration and Unity Catalog
• Strong knowledge of Apache Spark, PySpark, Python, and SQL
• Experience with Delta Lake and Lakehouse Architecture
• Azure Data Lake Storage (ADLS) and Azure Data Factory (ADF)
• Databricks Workflows, Job Orchestration, and ETL/ELT Pipelines
• Infrastructure as Code (Terraform) and CI/CD Automation
• Platform Engineering & Governance experience
• Technical Delivery & Leadership experience