Jobs → Data & Analytics → ETL Test Engineer / Azure Databricks Developer

About the Role We are seeking an ETL Test Engineer / Azure Databricks Developer to join our team in Sydney, Australia. This role involves performing end-to-end ETL/ELT testing, designing scalable data pipelines, and validating data quality and transformation logic. Key Responsibilities • Perform end-to-end ETL/ELT testing, source-to-target validation, data migration, and reconciliation • Design, develop, and optimize scalable data pipelines using Azure Databricks, Scala Spark, SQL, and Python • Validate data quality, transformation logic, business rules, KPIs, and reports • Work with Delta Lake and Medallion Architecture (Bronze, Silver, Gold) • Execute SIT, regression, end-to-end, and UAT testing • Develop SQL queries for database validation and data analysis • Support real-time and near-real-time data processing using Spark Streaming and Kafka • Implement job scheduling, CI/CD pipelines, monitoring, and defect tracking • Collaborate with Data Engineers, Business Analysts, and business stakeholders Requirements • 6–10 years of experience in ETL/ELT testing and data engineering • Hands-on experience developing and optimizing data pipelines using Azure Databricks • Strong practical experience with Apache Spark using Scala • Advanced SQL skills for complex queries, joins, data validation, reconciliation, and performance tuning • Python skills for data processing, validation, and automation • Experience with ETL/ELT & Data Warehouse Testing • Experience with Azure Data Engineering & Data Integration tools • Valid Australian visa with work rights Benefits • Opportunity to work with cutting-edge technologies in data engineering • Collaborative and innovative work environment