Azure Databricks Data Engineer
Shubham MishraRemoteSENIORPosted 13h ago
Azure DatabricksDatabricks WorkflowsAzure Data Lake StorageDelta LakeMedallion ArchitectureDatabricks NotebooksDelta Live TablesDatabricks SQL WarehouseData WarehousingDimensional ModelingUnity CatalogPySparkSQLPower BI Integration
About the Role
We are looking for an experienced Azure Databricks Data Engineer to join our team. This role involves building and maintaining end-to-end data pipelines and developing data ingestion frameworks.
Key Responsibilities
• Build and maintain end-to-end data pipelines using Databricks Workflows
• Develop data ingestion frameworks from SAP, CRM, databases, APIs & files
• Design and implement Bronze, Silver & Gold data layers using Delta Lake
• Develop data transformation workflows using PySpark, SQL, Databricks Notebooks & DLT
• Design analytical data models including Fact Tables, Dimension Tables & Star Schemas
• Load, optimize and maintain data in Databricks SQL Warehouse
• Implement governance, security, lineage and access controls using Unity Catalog
• Collaborate with Power BI teams for KPI, semantic model and reporting requirements
• Troubleshoot, tune and optimize Databricks workloads
• Work on migration from QlikView / ADF / Synapse / legacy BI platforms to Databricks
Requirements
• 6+ years of experience
• Azure Databricks
• Databricks Workflows
• Azure Data Lake Storage (ADLS)
• Delta Lake
• Medallion Architecture
• Databricks Notebooks
• Delta Live Tables (DLT)
• Databricks SQL Warehouse
• Data Warehousing & Dimensional Modeling
• Unity Catalog
• PySpark
• SQL
• Power BI Integration
• Certifications: DP-750, DCA-DL, DCP-PE