Lead Data Engineer
Shubham MaliIndiaLEADPosted 4h ago
DatabricksPythonSQLETLELTData ArchitectureData ModelingData IntegrationUnity CatalogREST APIsData QualityPipeline MonitoringCost Optimization
About the Role
Acute InfoSoft Pvt. Ltd. is seeking an experienced Lead Data Engineer with expertise in Databricks, Python, and SQL to develop scalable data pipelines.
Key Responsibilities
• Design and implement Databricks architecture, jobs, and workflows
• Develop and optimize ETL/ELT pipelines using Python and SQL
• Create data models and integrate data from various sources
• Implement data governance using Unity Catalog
• Develop REST APIs for data access
• Ensure data quality and reconciliation
• Monitor pipelines, troubleshoot issues, and optimize costs
Requirements
• 5–6 years of experience in data engineering
• Strong expertise in Databricks, Python, and SQL
• Experience with ETL/ELT pipeline development
• Knowledge of data architecture, modeling, and integration
• Familiarity with Unity Catalog and data governance
• Experience with REST APIs, data quality, and reconciliation
• Pipeline monitoring, troubleshooting, and cost optimization skills