Azure Databricks Data Engineer
Shaik MunnaRemoteSENIORPosted 1h ago
Azure DatabricksPySparkApache SparkSparkSQLAzure Data FactoryAzure Data LakeADLSGen2Delta LakeDelta Live TablesUnity CatalogPythonSQLETLELTData EngineeringData AnalyticsMedallion ArchitectureLakehouseAzure DevOpsPowerBI
About the Role
We are looking for an Azure Databricks Data Engineer to join our team. This role focuses on building scalable data solutions using Azure Databricks and related technologies.
Key Responsibilities
• Design and implement ETL/ELT processes using Azure Databricks
• Develop data analytics solutions with PySpark and SparkSQL
• Utilize Azure Data Factory, Azure Data Lake, and ADLSGen2 for data storage and processing
• Implement Delta Lake and Delta Live Tables for data management
• Work with Unity Catalog for data governance
• Use Python and SQL for data engineering tasks
• Apply Medallion Architecture and Lakehouse principles
• Utilize Azure DevOps for CI/CD
• Create data visualizations with PowerBI
Requirements
• Strong experience with Azure Databricks and PySpark
• Proficiency in Apache Spark and SparkSQL
• Experience with Azure Data Factory and Azure Data Lake
• Knowledge of Delta Lake and Delta Live Tables
• Familiarity with Unity Catalog and data governance
• Strong programming skills in Python and SQL
• Understanding of ETL and ELT processes
• Experience with data engineering and analytics
• Knowledge of Medallion Architecture and Lakehouse
• Experience with Azure DevOps and CI/CD
• Ability to create data visualizations with PowerBI