Jobs → Data & Analytics → Databricks Data Engineer

About the Role We are seeking an experienced Senior Databricks Data Engineer with strong hands-on expertise in Databricks, PySpark, Spark SQL, and AWS cloud technologies. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines and enterprise-grade data engineering solutions. Key Responsibilities • Design, develop, and maintain scalable data engineering solutions using Databricks, PySpark, Spark SQL, and Apache Spark. • Build, manage, and optimize end-to-end ETL/ELT pipelines integrating Databricks with AWS S3, Snowflake, Apache Airflow, and AWS Glue. • Process and transform large-scale structured and semi-structured datasets for enterprise reporting, analytics, and regulatory requirements. • Develop reusable frameworks for data ingestion, cleansing, transformation, validation, reconciliation, enrichment, and business rule implementation. • Implement data quality checks, monitoring, and governance controls to ensure data accuracy, consistency, and reliability. • Support customer segmentation, campaign analytics, AML and regulatory reporting, data validation, and reconciliation initiatives. • Participate in data migration, modernization, and continuous improvement projects. • Follow CI/CD, DevOps, and Agile/Scrum practices to ensure quality and reliable delivery. Requirements • Strong hands-on experience with Databricks and Lakehouse architecture. • Expertise in distributed data processing, transformations, and Spark performance tuning. • Strong SQL development, complex query writing, and data analysis skills. • Hands-on experience with AWS S3 and AWS Glue. • Experience integrating Databricks pipelines with Snowflake. • Experience developing and managing workflow orchestration and pipeline scheduling. • Proven experience designing and maintaining scalable data pipelines. • Experience handling JSON, Parquet, Avro, CSV, and other structured and semi-structured data formats. • Experience implementing data validation, reconciliation, monitoring, and quality control frameworks. • Strong understanding of data modeling, data warehousing, and distributed computing.