Lead Data Engineer
Abhishek BohidarUnited StatesLEADPosted 10d ago
PythonFastAPIAzureDatabricksPySparkSQLAzure FunctionsService BusAKSDomain-Driven DesignEvent-Driven ArchitecturePytestUnittestGitSVNCI/CDOOPSOLID principlesAgileScrumReactAngularTypeScriptApache KafkaMicroservices Architecture
About the Role
We are looking for a Lead Data Engineer to join our team. This role focuses on building scalable data platform services, developing production-grade APIs, and enabling reliable, AI-ready data solutions.
Key Responsibilities
• Design and develop scalable data platform services using Python, FastAPI, Azure AKS, and Databricks
• Build and integrate secure, reliable RESTful APIs
• Develop and optimize data-intensive applications using PySpark and SQL
• Work with Azure Function Apps, Azure Service Bus, and cloud-native technologies
• Drive technical design, code quality, automated testing, and CI/CD practices
• Mentor engineers and promote reusable components and platform engineering standards
• Troubleshoot production issues and improve system performance, reliability, and observability
• Collaborate with product managers, data scientists, and engineering teams to deliver scalable solutions
• Contribute to AI-ready data platforms and intelligent automation capabilities
Requirements
• Python, FastAPI, and RESTful APIs
• Microsoft Azure, Azure Functions, Service Bus, and AKS
• Databricks, PySpark, and SQL
• Domain-Driven Design (DDD) and Event-Driven Architecture
• Pytest, Unittest, and automated testing
• Git/SVN, CI/CD, OOP, and SOLID principles
• Agile/Scrum, production support, and technical leadership
• 10+ years of professional software development experience