Senior Databricks Data Engineer
Vivek PharswanUnited StatesSENIORPosted 7h ago
DatabricksPySparkApache SparkDelta LakeAWSS3IAMGlueLambdaRedshiftSQLPL/SQLBatch ProcessingReal-Time DataDataFramesHiveDistributed ComputingData ModelingData QualityPerformance TuningCI/CDGitDevOpsPython
About the Role
We are looking for a Senior Databricks Data Engineer to join our team. This role focuses on building scalable data platforms, modern Lakehouse architectures, and high-performance data pipelines.
Key Responsibilities
• Design and develop scalable end-to-end data engineering solutions
• Build and optimize data ingestion and transformation pipelines using Databricks and PySpark
• Develop and maintain conceptual, logical, and physical data models
• Lead High-Level Design (HLD) and Low-Level Design (LLD)
• Implement data quality, governance, monitoring, and performance optimization
• Collaborate with Product Owners, Architects, and Business Analysts
• Provide technical leadership and mentor junior engineers
Requirements
• 8+ years of experience in Data Engineering, ETL/ELT & Data Warehousing
• 5+ years of hands-on experience with Databricks, PySpark & Apache Spark
• Strong expertise in Delta Lake & Lakehouse Architecture
• Advanced SQL & PL/SQL development skills
• Experience building scalable Batch & Real-Time Data Pipelines
• Strong knowledge of Spark SQL, DataFrames, Hive & Distributed Computing
• AWS Services: S3, IAM, Glue, Lambda & Redshift
• Databricks-AWS Integration & Cloud-Native Architecture
• Data Modeling: Dimensional, Relational & Analytical
• Data Quality, Performance Tuning & Optimization
• CI/CD, Git, DevOps & Deployment Automation