Data Platform Engineer
Prasanna PRemoteSENIORPosted 2h ago
AzureADLSApache IcebergAzure KubernetesSparkDPTCI/CDDBTTrinoKubernetes
About the Role
We are looking for a Data Platform Engineer to join our team. This role focuses on designing, deploying, and operating data platform services on Kubernetes in Microsoft Azure.
Key Responsibilities
• Design, deploy, and operate data platform services on Kubernetes in Microsoft Azure
• Build and manage lakehouse infrastructure on Apache Iceberg backed by ADLS, including catalog configuration, table maintenance, schema evolution, and partitioning strategy
• Operate and tune Trino and Spark clusters for performance, reliability, and cost efficiency
• Develop and maintain data transformation pipelines using DBT, with sound modeling, testing, and documentation practices
• Define and enforce standards for data access, security, governance, and multi-tenant resource management across ADLS and query engines
• Implement observability for the platform, including monitoring, alerting, capacity planning, and incident response
• Partner with data engineers, analysts, and platform teams to onboard workloads and resolve performance bottlenecks
• Automate infrastructure and deployments using infrastructure-as-code and CI/CD
Requirements
• Experience with Azure, ADLS, Apache Iceberg, Azure Kubernetes (AKS), Spark, DPT, CI/CD
• Strong understanding of data platform services and Kubernetes
• Experience with data transformation pipelines and DBT
• Knowledge of observability, monitoring, and incident response
• Familiarity with infrastructure-as-code and CI/CD practices