Data Engineer
Sonu Kumar SahUnited StatesSENIORPosted 9h ago
DatabricksApache SparkPySparkDelta LakeDelta TablesETLELTSQLData IngestionData TransformationData Processing
About the Role
We are looking for an experienced Data Engineer with strong hands-on expertise in Databricks, Apache Spark, PySpark, and Delta Lake to build and maintain scalable, production-grade data engineering pipelines.
Key Responsibilities
• Design, develop, and maintain scalable data pipelines using Databricks
• Build production-ready ETL/ELT pipelines using Spark and PySpark
• Create, manage, and optimize Delta tables
• Develop and schedule Databricks Jobs and Workflows
• Optimize Spark jobs for performance, scalability, and cost efficiency
• Implement data validation, error handling, and quality checks
• Monitor production pipelines and resolve data processing issues
• Collaborate with data architects, analysts, and engineering teams
Requirements
• 7+ years of overall IT experience
• Proven hands-on experience building and deploying pipelines on Databricks
• Production experience with Spark/PySpark and Delta Lake
• Experience developing and managing Databricks Jobs and Workflows
• Strong pipeline troubleshooting and performance optimization skills