Jobs → Data & Analytics → Lead Data Engineer

About the Role We are seeking a Lead Data Engineer to build and run our data platform, provide operational teams with trusted data, and lead a small team of experienced analysts. Key Responsibilities • Design, build and maintain Azure Data Factory pipelines and the SQL estate, with a focus on reliability, monitoring and performance • Develop data models that support reporting, analysis and self-service use • Manage ingestion of external data from clients and applications, including data quality checks and error handling • Support the adoption of Apache Superset for Group reporting alongside Power BI • Document pipelines, data models and data sources so they can be supported by the wider team • Put all pipelines, datasets and reports under version control, with CI/CD deployment across development, test and production • Coach and develop the team's engineering skills Requirements • Significant hands-on data engineering experience, including leading or mentoring a small team • Strong Azure Data Factory and SQL Server / Azure SQL, including pipeline design, data modelling and performance tuning • Power BI, including data modelling, workspace management and deployment pipelines • Version control and CI/CD for data pipelines and reports (Git, Azure DevOps or GitHub) • Integrating varied external data sources (APIs, flat files, client extracts), with data quality checks and error handling • Building trusted datasets that support self-service reporting and performance analysis • UK GDPR and the Data Protection Act 2018 Desirable • Python for data transformation or automation • Azure Synapse, Fabric or Databricks • Infrastructure as code, such as Bicep or Terraform • Using AI tools in data work and preparing data for AI use, such as Azure AI or Copilot • Apache Superset, or rolling out a new BI tool • Public sector or government-funded contract environments, such as employment, skills or education services • Awareness of ICO guidance on AI and data protection • Microsoft data certification, such as Fabric Data Engineer (DP-700) or PL-300