Databricks Administrator
Emmanuel RubenUnited StatesSENIORPosted 3h ago
DatabricksApache SparkPySparkDelta LakeLakehouse ArchitectureETL/ELTUnity CatalogRBACAzure DatabricksAWS DatabricksADLSS3IAMAzure Key VaultPythonDatabricks CLIREST APIsTerraformGit
About the Role
We are actively hiring a Databricks Administrator to manage, configure, secure, and optimize enterprise Databricks environments for large-scale data engineering, analytics, and AI/ML workloads.
Key Responsibilities
• Administer and maintain Databricks workspaces, compute resources, clusters, jobs, and workspace configurations
• Configure workspace settings, access permissions, cluster policies, service principals, and user provisioning
• Implement Unity Catalog, role-based access controls, data permissions, auditing, and secure access to enterprise data
• Manage integrations with cloud storage, identity providers, networking, secret management, and cloud security services
• Monitor compute utilization, optimize cluster configurations, implement autoscaling, and control platform costs
• Diagnose Spark job failures, cluster startup issues, slow queries, pipeline failures, and resource bottlenecks
• Automate administrative tasks using Python, Databricks CLI, REST APIs, and Terraform
• Configure scheduled jobs, workflows, alerts, dependencies, and failure-recovery mechanisms
• Support development, testing, and production deployments using Git-based workflows and CI/CD pipelines
Requirements
• Strictly 8+ years of experience with Databricks administration and engineering
• Strong knowledge of Apache Spark, PySpark, Delta Lake, and Lakehouse Architecture
• Experience with ETL/ELT pipelines and data engineering concepts
• Expertise in Unity Catalog, RBAC, access control, and data governance
• Experience with Microsoft Azure Databricks, AWS Databricks, or Databricks on Google Cloud
• Familiarity with Azure Data Lake Storage (ADLS), AWS S3, IAM, and Azure Key Vault
• Proficiency in Python, Databricks CLI, REST APIs, and Terraform
• Experience with Git-based workflows and CI/CD pipelines