Data Engineer
Aditya SharmaUnited StatesSENIORPosted 12d ago
PySparkSQLDelta LakeUnity CatalogDatabricks WorkflowsAzure DatabricksADLS Gen2Key Vault
About the Role
We are looking for an experienced Data Engineer to join our team to build production lakehouse pipelines and GenAI-ready data products on Azure Databricks.
Key Responsibilities
• Build pipelines using PySpark, SQL, Delta Lake, Unity Catalog, and Databricks Workflows
• Design bronze, silver, and gold layers with clear data quality checks and schemas
• Create data pipelines for ML and GenAI, including embeddings, RAG chunking, and feature tables
• Integrate with Azure services like ADLS Gen2 and Key Vault
Requirements
• 5+ years in data engineering, with 2+ years on Azure Databricks
• Strong Python, PySpark, and SQL skills
• Hands-on experience with Delta Lake and Unity Catalog
• Experience building data pipelines for ML or GenAI applications
• Valid independent work authorization; no visa sponsorship offered