Data Engineer
Meenakshi M.United StatesMIDPosted 12d ago
Azure DatabricksPythonPySparkSQLDelta LakeUnity CatalogData pipelinesGenAIRAGembeddingschunkingvector searchGitCI/CDunit testingintegration testingAzure servicesADLS Gen2Key VaultManaged Identities
About the Role
We are looking for experienced Data Engineers with strong Azure Databricks and GenAI data platform expertise for a contract opportunity.
Key Responsibilities
• Design and implement data pipelines using Azure Databricks
• Develop and maintain GenAI solutions including RAG, embeddings, and vector search
• Utilize Delta Lake and Unity Catalog for data management
• Implement CI/CD practices and conduct unit and integration testing
• Work with Azure services like ADLS Gen2, Key Vault, and Managed Identities
Requirements
• 5+ years of experience in Data Engineering
• 2+ years of production experience with Azure Databricks
• Proficiency in Python, PySpark, and SQL
• Experience with data pipelines and Bronze/Silver/Gold architecture
• Knowledge of GenAI, RAG, embeddings, chunking, and vector search
• Familiarity with Git, CI/CD, and testing practices