Data Engineer
SiDRAFORCE SolutionsRemoteSENIORPosted 11d ago
Azure DatabricksData GovernanceSODAAlationPythonPySparkData Pipeline DevelopmentAzureAWSGCPKubernetesTerraformSQLDatabase ManagementData IngestionIntegration TestingCI/CD PipelinesStatic Code AnalysisData Security
About the Role
We are looking for a Data Engineer to join our team. This role focuses on developing and maintaining data pipelines and ensuring data governance.
Key Responsibilities
• Develop and maintain data pipelines using Azure Databricks and PySpark
• Implement data governance policies using tools like Alation and SODA
• Work with cloud platforms including Azure, AWS, and GCP
• Manage Kubernetes and Terraform for infrastructure
• Ensure data security and compliance
Requirements
• 8+ years of experience in data engineering
• Strong skills in Python, PySpark, and SQL
• Experience with data pipeline development and CI/CD pipelines
• Knowledge of data governance and security practices
• Familiarity with Kubernetes, Terraform, and cloud platforms