Databricks Governance Engineer
Abhishek DevdharUnited StatesSENIORPosted 10d ago
SQLDatabricksCollibraAWSdata profilingPentahoAWS GlueAthenaRedshiftLake FormationDatabricks Unity CatalogPythonPySpark
About the Role
We are looking for a skilled Databricks Governance Engineer to support enterprise data governance, metadata management, data quality, and reconciliation initiatives across cloud platforms and enterprise data environments.
Key Responsibilities
• Discover, profile, onboard, and catalog enterprise data assets across databases, data lakes, and AWS S3.
• Perform data profiling, reconciliation, and quality assessments to identify inconsistencies and anomalies.
• Develop SQL-based queries and frameworks for record-level, attribute-level, and aggregate-level reconciliation.
• Support metadata management, data classification, lineage, stewardship, ownership, and business glossary initiatives.
• Work with Databricks, Collibra, Pentaho, and related tools to improve data cataloging and governance.
• Support automated metadata scanning, lineage tracking, and data classification.
• Analyze schemas, tables, files, and pipelines to identify data quality and governance gaps.
• Collaborate with data engineers, architects, data owners, stewards, and business stakeholders to resolve data issues.
Requirements
• Strong SQL and data reconciliation expertise
• Hands-on experience with data governance, metadata management, and data quality
• Experience with Databricks or similar lakehouse platforms
• Experience with Collibra or comparable data catalog and governance tools
• Working knowledge of AWS, especially Amazon S3
• Understanding of data lineage, classification, stewardship, ownership, and business glossaries
• Strong analytical, problem-solving, documentation, and communication skills