Data Engineer
Praveen KumarRemoteMIDPosted 14d ago
SQLPythonPySparkAWS S3AWS GlueAWS AthenaAWS Lake FormationAWS Step FunctionsAWS LambdaLarge-scale data processingData pipeline developmentAPIBatch ingestionData qualityData reconciliationData modellingData partitioningMetadata managementData lineageIdentity mappingData preparationAnalyticsMLSnowflakeDatabricks
About the Role
We are looking for an experienced Data Engineer with strong hands-on expertise in SQL, Python, PySpark, and AWS data services.
Key Responsibilities
• Design and implement data pipelines
• Develop and maintain ETL processes
• Work with AWS data services including S3, Glue, and Athena
• Ensure data quality and reconciliation
• Build data models and manage metadata
• Prepare data for analytics and machine learning
Requirements
• 5–7 years of experience as a Data Engineer
• Strong hands-on expertise in SQL, Python, and PySpark
• Experience with AWS data services (S3, Glue, Athena, Lake Formation, Step Functions, Lambda)
• Experience with large-scale data processing and data pipeline development
• Experience with API and batch ingestion
• Experience with data quality, reconciliation, modelling, and partitioning
• Experience with metadata management and data lineage
• Experience with identity mapping
• Experience with data preparation for analytics and ML
• Mandatory: AWS certification through the official AWS platform
Good to Have
• Experience with Snowflake or Databricks
• Domain experience in Commerce, Retail, Media, or CPG