Loading...
Mid LevelContractPythonAzureElasticsearchAIML
Salary Range
$35,700 - $48,300 /year
This salary is estimated based on similar roles. The actual salary may vary.
About the Role
The Senior Data Engineer will design, build, and optimize batch data pipelines and algorithms that power a matching engine for a leading global management consultancy. This hands-on role focuses on scale, performance, and accuracy, working with Spark, SQL, and modern orchestration tools.
Key Responsibilities
- →Build and maintain robust, scalable ETL/ELT pipelines to ingest and process third-party and first-party datasets
- →Apply transformation, normalization, and enrichment rules to ensure data consistency and usability
- →Collaborate with product managers, data architects, and content experts to align data structure with business needs
- →Support implementation of data matching and entity resolution processes using AI/ML tools
- →Build alerts, logs, and metrics to ensure data flows remain healthy
- →Contribute to documentation, code quality standards, and internal best practices
Requirements
- ✓5+ years of experience in Big Data, Data Platform, or ETL engineering roles
- ✓Proficient in SQL, Python, and experience with Spark(PySpark or Scala), Airflow, Snowflake, and Azure Data Lake
- ✓Familiarity with Azure or other major cloud platforms
- ✓Proven experience designing and operating large-scale batch data pipelines
- ✓Solid understanding of distributed systems and algorithms
- ✓Proactive, detail-oriented, and eager to take ownership of projects
- ✓Comfortable working in a cross-functional environment
