Data Engineer
Sneha NRemoteSENIORPosted 12d ago
Azure DatabricksData PipelinesData IngestionData GovernanceSODAAlationPythonPySparkAzureAWSGCPKubernetesTerraformRelational DatabasesBig Data DatabasesUnit TestingIntegration TestingCI/CD PipelinesStatic AnalysisCode Quality ToolsLarge-Scale Data Processing
About the Role
We are seeking a Data Engineer to join our team and contribute to building scalable data processing systems. This role requires expertise in data pipelines, governance, and cloud platforms.
Key Responsibilities
• Design, develop, and optimize data pipelines
• Build and maintain scalable data processing systems
• Ensure data quality, security, governance, and compliance
• Collaborate with architects and cross-functional teams
• Troubleshoot issues and support production-ready deployments
Requirements
• 8+ years of experience as a Data Engineer
• Hands-on expertise with Azure Databricks
• Experience in data governance using SODA and Alation
• Strong skills in Python and PySpark
• Knowledge of Azure, AWS, or GCP
• Experience with Kubernetes and Terraform
• Familiarity with relational and big data databases
• Experience with unit and integration testing
• Understanding of CI/CD pipelines
• Knowledge of static analysis and code quality tools
• Experience with large-scale data processing