Lead Data Engineer
Shailja PorwaalUnited StatesLEADPosted 12d ago
Data EngineeringDatabricksApache SparkDelta LakeSQLPythonPySparkAzure Cloud ServicesAzure Data FactoryADLSEvent HubsAzure FunctionsReal-time streamingIoT telemetryData orchestrationDevOpsCI/CDData qualityMonitoringProduction troubleshootingData servicesAPIsMicroservicesScalable data productsPerformance tuningCost optimizationData governanceProduction reliability
About the Role
We are seeking a Lead Data Engineer to join our team in Philadelphia, PA. This role involves hands-on experience in data engineering, cloud services, and leadership responsibilities.
Key Responsibilities
• Design and implement data engineering solutions using Databricks, Apache Spark, and Delta Lake
• Develop real-time streaming pipelines and IoT telemetry solutions
• Build and maintain data pipelines and scalable data products
• Lead technical architecture and mentor junior team members
• Manage stakeholder relationships and ensure production reliability
Requirements
• 10+ years of experience in Data Engineering
• Expertise in Azure Cloud Services and related platforms
• Strong hands-on experience with SQL, Python/PySpark, and Azure Data Factory
• Experience with data orchestration, DevOps, CI/CD, and data quality
• Leadership skills in technical architecture, team mentoring, and stakeholder management
• Experience with performance tuning, cost optimization, and data governance