Data Engineer
Rajeev ZUnited StatesMIDPosted 6d ago
DatabricksPySparkApache SparkDelta LakeETL/ELTData PipelinesCloud SecurityCost OptimizationUnity CatalogCI/CD
About the Role
We are looking for a Data Engineer to join our team. This role focuses on building and maintaining data pipelines using Databricks and related technologies.
Key Responsibilities
• Design and implement data pipelines using Databricks, PySpark, and Apache Spark
• Work with Delta Lake for data lakehouse architecture
• Develop ETL/ELT processes for data transformation
• Ensure data quality and implement data security measures
• Optimize cloud costs and implement CI/CD for data workflows
Requirements
• 3+ years of experience in data engineering
• Strong skills in Databricks, PySpark, and Apache Spark
• Experience with Delta Lake, ETL/ELT, and data pipelines
• Knowledge of cloud security and cost optimization
• Familiarity with Unity Catalog and CI/CD practices