Data Engineer
Putul MahatoRemoteMIDPosted 14h ago
SparkPySparkSQLPythonJavaScalaAzureAWSGCPSnowflakeSynapseAzure DevOpsMLOpsAzure Data FactoryDatabricks
About the Role
We are looking for a Data Engineer to join our team and build and modernize large-scale data solutions on Azure. This role involves working on terabyte-scale pipelines, hybrid architectures, and distributed systems that make a real impact.
Key Responsibilities
• Design and build Azure data pipelines and streaming solutions (on-premise + cloud hybrid)
• Support Solution Architects in data transformation and modernization programs
• Build tools to ingest and process terabyte-scale data daily
• Write production-quality code to deploy ML workflows in distributed environments
• Troubleshoot complex distributed system issues
• Prepare design documents, technical specs, and test plans
• Own development, production deployment, post-prod support, and support UT, SIT & UAT
• Collaborate with product owners, architects, business analysts, platform engineers, and cloud vendors
Requirements
• Strong hands-on experience in Spark/PySpark
• Expertise in SQL and performance optimization
• Proficiency in Python (preferred), Java, or Scala
• Experience with cloud platforms: Azure (preferred), AWS, or GCP
• Data warehousing with Snowflake or Synapse, including dimensional modeling for DWH and data marts
• Knowledge of Azure DevOps, version control, build automation, and/or MLOps
• Understanding of OS internals, file systems, storage technologies, and protocols
Good to have: Azure Data Factory (ADF), Databricks