Databricks Admin
Chetan ManglaUnited StatesSENIORPosted 1h ago
DatabricksAzure/AWSSparkUnity CatalogDelta LakeCluster ManagementSecurityRBACPerformance OptimizationAzure CloudADLS Gen2Azure DevOpsTerraformGitCI/CDPythonPowerShellBash
About the Role
We are seeking an experienced Databricks Administrator to join our team. This role focuses on managing enterprise Databricks environments and ensuring platform stability, security, and performance.
Key Responsibilities
• Administer and manage enterprise Databricks workspaces across Development, QA, UAT, and Production environments
• Configure, deploy, and maintain Databricks clusters, cluster policies, instance pools, and compute resources
• Monitor platform health, performance, availability, and capacity planning
• Implement security best practices, including RBAC, Unity Catalog, workspace permissions, secret scopes, and identity management
• Configure and administer Unity Catalog, data governance, access control, and metadata management
• Integrate Databricks with Azure Data Lake Storage (ADLS Gen2), Azure Key Vault, AWS S3, or cloud storage services
• Optimize Spark cluster performance, auto-scaling, job execution, and resource utilization
• Manage Databricks Jobs, Workflows, notebooks, and scheduling
• Support CI/CD implementation using Azure DevOps, GitHub, GitLab, or Jenkins
• Perform platform upgrades, patching, backup, disaster recovery, and environment migrations
• Troubleshoot production issues related to clusters, notebooks, jobs, connectivity, and permissions
• Work closely with Data Engineers, Data Scientists, Cloud Engineers, and DevOps teams to support enterprise data platforms
• Develop automation scripts using Python, PowerShell, Bash, or Terraform
• Maintain operational documentation, runbooks, and standard operating procedures
Requirements
• 8+ years of overall IT experience
• 3–5+ years of hands-on Databricks Administration
• Strong experience administering Azure Databricks in enterprise environments
• Expertise in Unity Catalog, Delta Lake, Cluster Management, Security, RBAC, and Performance Optimization
• Hands-on experience with Azure Cloud, ADLS Gen2, Azure DevOps, Terraform, Git, and CI/CD pipelines
• Proven experience supporting large-scale production environments
• Experience working in Agile environments and supporting Data Engineering and AI/ML teams