Jobs → Data & Analytics → Databricks Data Engineer

About the Role We are looking for an experienced Databricks Data Engineer to join our team. This role involves developing and maintaining data pipelines and notebooks in Databricks, integrating data from various platforms, and ensuring data quality and reliability. Key Responsibilities • Develop and maintain notebooks in Databricks to extract data from internal and external platforms. • Build, maintain, and optimize ETL/ELT pipelines within Databricks. • Integrate data from business applications, APIs, databases, and other enterprise platforms. • Use Python, SQL, REST APIs, JSON, and authentication mechanisms to retrieve, transform, and write data across multiple systems. • Work with Microsoft Graph API and other platform APIs for cross-system data exchange. • Prepare, clean, validate, transform, and organize data for reporting and analytics. • Support structured data-layer development, including raw, cleansed, and curated datasets. • Troubleshoot pipeline failures, API errors, data-quality issues, and performance bottlenecks. • Implement appropriate logging, error handling, validation, and documentation for integration workflows. • Collaborate with Power BI developers and business stakeholders to provide reliable data for dashboards and reporting. • Follow secure practices for credentials, API permissions, sensitive data, and access management. Requirements • Hands-on experience with Python programming and SQL. • Practical experience developing notebooks and data pipelines in Databricks, or demonstrably strong experience with comparable data engineering platforms. • Understanding of ETL/ELT concepts, data transformation, data modelling, and data-quality controls. • Experience integrating systems through REST APIs, JSON, and authentication mechanisms such as OAuth 2.0. • Experience with web technologies such as HTML, CSS, JavaScript, and relevant frameworks. • Ability to read from and write to databases, APIs, and external business platforms. • Understanding of scheduling, monitoring, error handling, and troubleshooting data pipelines. • Ability to analyse technical requirements and deliver end-to-end solutions with minimal supervision.