Jobs → Data & Analytics → Senior Data Engineer

About the Role We are looking for an experienced Senior Data Engineer to join an enterprise platform team, building scalable, secure, and reliable data pipelines to support regulatory controls, analytics, reporting, and AI/ML use cases. Key Responsibilities • Design, develop, and maintain scalable data pipelines using Python, PySpark, and Databricks. • Ingest, transform, validate, and onboard enterprise datasets. • Build data quality, completeness, and integrity controls. • Develop algorithms to identify anomalies in enterprise communications data. • Optimise data processing performance, reliability, and scalability. • Implement CI/CD, automated testing, monitoring, and SDLC best practices. • Leverage enterprise-approved AI coding assistants while ensuring code quality, security, and compliance. • Collaborate with stakeholders to deliver production-ready data engineering solutions. Requirements • Strong expertise in Python, SQL, and PySpark. • Experience with Databricks and data engineering practices. • Knowledge of data pipelines, ELT, and data transformation. • Familiarity with cloud platforms such as AWS and Azure. • Understanding of Delta Lake, Lakehouse, and Medallion Architecture. • Experience with CI/CD, testing, and deployment. • Skills in data quality, integrity, and performance optimisation. • Experience with AI-assisted software development tools. • Ideal candidate should have experience in enterprise data platforms and regulatory controls.