Azure Databricks Data Engineer
Abhilash KumarUnited StatesMIDPosted 2h ago
Azure CloudAzure DatabricksPySparkAzure Data FactoryDelta LakeUnity CatalogETL/ELTData ModelingPerformance TuningData GovernancePythonSQLGitHubCI/CDLarge-Scale DatasetsREST APIsGraphQLKubernetesAKSAPI SecurityAPI ContractsError HandlingAI/MLLLMAgentic FrameworksDatabricks AICodexClaude
About the Role
We are seeking a Data Engineer with strong hands-on experience in Azure Cloud and Azure Databricks to join our team. This role focuses on designing, developing, and optimizing scalable data pipelines and cloud-based data solutions.
Key Responsibilities
• Design, develop, and optimize scalable data pipelines and cloud-based data solutions
• Implement ETL/ELT processing, data modeling, and performance tuning
• Ensure data governance and security practices are followed
• Build REST or GraphQL APIs and deploy them using Kubernetes
• Work with large-scale datasets in enterprise environments
• Collaborate with cross-functional teams and business stakeholders
Requirements
• 6-12 years of experience in Azure Cloud and Azure Databricks
• Proficiency in PySpark, Azure Data Factory, Delta Lake, and Unity Catalog
• Strong experience in ETL/ELT processing, data modeling, and performance tuning
• Expertise in Python, SQL, GitHub, and CI/CD practices
• Experience with large-scale datasets in enterprise environments
• Understanding of AI/LLM or agentic frameworks
• Strong experience building REST or GraphQL APIs
• Experience with Kubernetes for API deployment
• Understanding of API security, contracts, error handling, and performance
• Strong understanding of data governance, security, and CI/CD practices