Databricks Architect – Azure
SURENDRA .MUnited StatesSENIORPosted 2h ago
Azure DatabricksPySparkDelta LakeUnity CatalogData EngineeringApache SparkPythonScalaSQLAzure Data Lake StorageAzure Blob StorageAzure DevOpsCI/CDGitApache KafkaAzure Event HubsAirflowAzure Data FactoryPower BIMLflowTerraform
About the Role
We are looking for an experienced Databricks Architect – Azure to join our team at EVERSOFT for our client, Deloitte. The ideal candidate will have strong expertise in Azure Databricks, PySpark, Delta Lake, and Unity Catalog, along with proven experience leading large data engineering teams.
Key Responsibilities
• Lead and mentor a team of 10+ Data Engineers
• Own end-to-end Databricks Lakehouse architecture and implementation
• Design and optimize data pipelines using PySpark, Apache Spark, and Delta Lake
• Implement Databricks Unity Catalog for data governance, security, and data lineage
• Manage Azure Databricks clusters, cluster policies, auto-scaling, and cost optimization
• Optimize Spark jobs, SQL queries, and ETL pipelines for performance
• Implement CI/CD pipelines using Azure DevOps and Git
• Ensure data security, encryption, masking, and compliance
• Collaborate with application developers, data analysts, and data scientists
• Drive platform innovation and establish data engineering best practices
Requirements
• 10+ years of experience in Data Engineering / Data Architecture
• Strong hands-on experience with Azure Databricks
• Expertise in Apache Spark, PySpark, Python, Scala, and Delta Lake
• Strong experience with Databricks Unity Catalog
• Advanced SQL and data warehousing concepts
• Experience with Azure Data Lake Storage (ADLS Gen2) and Azure Blob Storage
• Strong knowledge of Azure cloud services
• Experience with cluster management and performance tuning
• Hands-on experience with Azure DevOps, CI/CD, and Git
• Proven experience leading and mentoring data engineering teams