Databricks Administrator
Jagadesh ThotaSENIORPosted 11d ago
Databricks Workspace AdministrationAzure DatabricksAWS DatabricksDatabricks on GCPUnity CatalogData GovernanceCluster ManagementCompute ManagementSQL Warehouse ManagementJobsWorkflowsNotebooksIdentity and Access Management (IAM)SSOSCIMService PrincipalsApache SparkPySparkDelta LakeLakehouse ArchitectureGitCI/CDDevOpsMonitoringTroubleshootingPerformance TuningCloud SecurityCost Optimization
About the Role
We are seeking an experienced Databricks Administrator to manage, secure, optimize, and support enterprise Databricks environments across cloud platforms.
Key Responsibilities
• Administer Databricks workspaces across AWS, Azure, or Google Cloud.
• Manage clusters, compute policies, jobs, workflows, SQL warehouses, and workspace configurations.
• Configure Unity Catalog, catalogs, schemas, tables, permissions, and data governance policies.
• Manage users, groups, service principals, SSO, SCIM provisioning, and role-based access controls.
• Implement secure data access, secrets management, encryption, and compliance controls.
• Monitor platform usage, job failures, performance, costs, and resource utilization.
• Troubleshoot cluster startup issues, job failures, connectivity, and performance bottlenecks.
• Manage notebooks, libraries, runtime versions, repositories, and environment configurations.
• Support Delta Lake, Lakehouse architecture, and data engineering workloads.
• Automate administrative tasks using APIs, CLI tools, Terraform, and scripting.
• Implement workspace provisioning, configuration management, and infrastructure automation.
• Support CI/CD, Git integration, deployment workflows, and environment promotion.
• Establish platform standards, operational documentation, backup strategies, and recovery procedures.
• Collaborate with data engineers, architects, security teams, and cloud infrastructure teams.
Requirements
• Databricks Workspace Administration
• Azure Databricks / AWS Databricks / Databricks on GCP
• Unity Catalog / Data Governance
• Cluster / Compute / SQL Warehouse Management
• Jobs / Workflows / Notebooks
• Identity and Access Management (IAM)
• SSO / SCIM / Service Principals
• Apache Spark / PySpark
• Delta Lake / Lakehouse Architecture
• Git / CI/CD / DevOps
• Monitoring / Troubleshooting / Performance Tuning
• Cloud Security / Cost Optimization