Data Engineer / Architect
Rajat UpadhyayRemoteSENIORPosted 8h ago
PythonPySparkSQLdimensional modelingAIDPDatabricksSnowflakeAgentic AI frameworksPHI/HIPAA complianceOCIGoldenGateData MeshTerraform
About the Role
We are looking for a Data Engineer / Architect to join our team and help turn our Enterprise Data Lake (EDL) vision into low-level designs. This role involves driving end-to-end delivery across onshore/offshore teams using AI-native workflows.
Key Responsibilities
• Translate EDL architecture into actionable designs, work packages, and Gold-layer semantic models
• Build daily with AI coding tools (Claude Code, Codex) by writing markdown specs, skills, and context files for production code
• Architect Lakehouse pipelines (Python, PySpark, Delta Lake, Oracle GoldenGate CDC) on OCI/AIDP, Databricks, or Snowflake
• Enable Agentic AI data access via MCP-style interfaces, REST services, and canonical data products
• Enforce Healthcare governance (PHI/HIPAA), masking, Terraform, and CI/CD standards
Requirements
• 8+ years in Data Engineering/Architecture (2+ years leading teams)
• 1+ year building production systems with AI coding tools (Claude Code/Codex)
• Expertise in Python, PySpark, SQL, dimensional modeling, and AIDP/Databricks/Snowflake
• Experience with Agentic AI frameworks (MCP servers, knowledge graphs) and PHI/HIPAA compliance
Benefits
• Opportunity to work on cutting-edge AI-native workflows
• Chance to contribute to healthcare data governance and compliance