Data Platform Engineer
Siddhartha ChamoliUnited StatesSENIORPosted 1h ago
DatabricksSnowflakeAirflowMicrosoft AzureSQLPythonData modelingETL/ELTDistributed data processingRefactoringStandardizationOptimizationAI-assisted automation tools
About the Role
We are seeking a Data Platform Engineer to join our team for a 12-month contract. This role involves assessing and improving an existing data platform or designing new data solutions from the ground up.
Key Responsibilities
• Assess and improve an existing data platform, including refactoring pipelines, optimizing performance, and improving code quality
• Design, build, and maintain scalable data pipelines to support downstream application and analytics use cases
• Implement and manage data ingestion, transformation, and storage patterns using Databricks and Snowflake
• Partner with architects and stakeholders to define data models and schemas
• Ensure data quality, reliability, observability, and governance across the data lifecycle
• Optimize workloads for cost, performance, and scalability in a cloud native Azure environment
• Contribute to and enforce engineering standards, including documentation, testing, version control, and CI/CD practices
• Support and enable downstream consumers by delivering trusted, well-structured data assets
Requirements
• Strong experience as a Data Engineer building and maintaining production grade data platforms
• Proficiency with Databricks (Spark, Delta Lake, notebook)
• Proficiency with Airflow
• Proficiency with Snowflake
• Experience building data solutions in Microsoft Azure
• Proficiency in SQL
• Proficiency with Python
• Solid understanding of data modeling, ETL/ELT patterns, and distributed data processing
• Experience improving existing codebases through refactoring, standardization, and optimization
• Strong collaboration and communication skills
• Experience using AI-assisted automation tools to improve efficiency and productivity
Desired skills
• Familiarity with data governance, metadata management, or data quality frameworks
• Experience supporting application-centric data use cases
• Exposure to DevOps or DataOps practices (CI/CD, infrastructure as code, monitoring)
• Experience working in large, regulated, or enterprise scale environments
• Experience working with data that contains PHI