Agentic Data Engineer
Abhinav ShivhareUnited StatesMIDPosted 11d ago
Agentic Data EngineeringAI AutomationIntelligent workflowPipeline orchestrationData cleansingData quality remediationException managementHuman-in-the-loop workflowsAI-driven data engineeringDatabricks ArchitectureLakehouse ArchitectureMedallion ArchitectureDelta LakeUnity CatalogPipeline developmentDatabricks WorkflowsABACData lineageData governanceRow-level securityEnterprise security frameworksPythonGitCI/CDData pipelinesData modelingETL/ELTCloud platformsAWSEnterprise data integrationData transformation
About the Role
We are seeking an experienced Agentic Data Engineer to build intelligent, AI-enabled data products and automated data engineering solutions. This role focuses on enterprise data domains, unified customer data, intelligent pipeline orchestration, and reducing manual intervention through AI-driven workflows.
Key Responsibilities
• Build unified enterprise data domains covering customers, accounts, relationships, risk, and operational activities
• Implement data mastering, hierarchy modeling, and cross-domain integration
• Develop AI-enabled automation and intelligent pipeline orchestration
• Manage data cleansing and quality remediation
• Implement exception management and human-in-the-loop workflows
Requirements
• Experience with Agentic Data Engineering & AI Automation
• Expertise in Databricks Architecture, Lakehouse & Medallion Architecture
• Knowledge of Delta Lake and Unity Catalog
• End-to-end pipeline development capabilities
• Familiarity with Databricks Workflows and ecosystem
• Understanding of Data Governance & Security principles
• ABAC (Attribute-Based Access Control) experience
• Data lineage, governance, and auditability knowledge
• Row-level security and entitlement-based architectures experience
• Python, Git, and CI/CD skills
• Data pipelines and modern data engineering practices knowledge
• Data modeling and ETL/ELT experience
• Cloud platforms, preferably AWS
• Enterprise data integration and transformation experience