AWS Data Engineer
Bharath CHUnited StatesMIDPosted 12d ago
DatabricksAWSTerraformUnity CatalogIAMVPCS3KMSSecrets ManagerCloudTrailPrivateLinkVPC peeringTransit gatewaysSecurity groupsNACLsDatabricks WorkflowsCLISDKReposGit integrationGitHub ActionsDatadogGrafana
About the Role
We are looking for an AWS Data Engineer with production Databricks experience to join our team. This role focuses on designing and administering Databricks workspaces on AWS.
Key Responsibilities
• Design and implement RESTful APIs
• Lead code reviews and mentor junior developers
• Collaborate with product team on technical requirements
Requirements
• 6+ years of cloud platform or data infrastructure engineering experience
• 2+ years of production Databricks experience
• Strong hands-on experience designing and administering Databricks workspaces on AWS
• Deep AWS infrastructure experience with VPC, IAM roles and policies, cross-account access, SCPs, S3, KMS, Secrets Manager, and CloudTrail
• Strong Terraform experience, specifically with the Databricks Terraform provider
• Hands-on expertise with Unity Catalog administration
• Experience implementing column-level masking, row-level security, dynamic views, lineage, and audit logging
• Strong knowledge of private networking
• Hands-on experience defining Databricks cluster policies
• Experience with Databricks Workflows, system tables, CLI, SDK, Repos, and Git integration
• Strong CI/CD experience using GitHub Actions
• Experience implementing platform monitoring, cost attribution, and observability
• Ability to create platform architecture documentation, operational runbooks, onboarding standards, and production support procedures