Data Engineer
Renuka sonarRemoteMIDPosted 24m ago
AWS GlueRedshiftS3LambdaAzure Data FactoryDatabricksPythonPySparkAdvanced SQLAzure/SQL ServerAWS Data MigrationAPI-basedCloud Data SourcesApache IcebergHudiDelta LakeAWS DevOpsCI/CD PipelinesGDPRPII Data SecurityData MaskingAnonymizationPseudonymizationTroubleshootingPerformance Optimization
About the Role
We are looking for a Data Engineer with 5+ years of experience to join our team. This role focuses on AWS data engineering and Azure-to-AWS migration.
Key Responsibilities
• Design and implement data solutions using AWS Glue, Redshift, S3, and Lambda
• Migrate data from Azure Data Factory and Databricks to AWS
• Develop with Python, PySpark, and Advanced SQL
• Implement API-based and cloud data sources
• Utilize Apache Iceberg, Hudi, and Delta Lake for data lakehouse solutions
• Ensure GDPR and PII data security compliance
• Perform data masking, anonymization, and pseudonymization
• Troubleshoot and optimize performance
Requirements
• 5+ years of experience in data engineering
• Strong hands-on expertise in AWS data engineering
• Experience with Azure-to-AWS migration
• Proficiency in Python, PySpark, and Advanced SQL
• Knowledge of Azure Data Factory and Databricks
• Understanding of API-based and cloud data sources
• Familiarity with Apache Iceberg, Hudi, and Delta Lake
• Experience with AWS DevOps and CI/CD Pipelines
• Understanding of GDPR and PII Data Security
• Skills in data masking, anonymization, and pseudonymization
• Ability to troubleshoot and optimize performance