Loading...
Mid LevelFull-timePythonGoAzureAICI/CD
About the Role
We are seeking a Data Engineer to join our team for a niche financial project focused on Customer Registration Transformation management. This role involves developing and maintaining data pipelines in Azure environments while collaborating with business and data science teams.
Key Responsibilities
- →Develop and maintain data pipelines (ETL/ELT) in Azure using Databricks for processing and integration
- →Implement CI/CD pipelines for ETL processes
- →Create data ingestion, transformation, and modeling solutions using Databricks, Spark/PySpark, Cloudera, and Azure Data Factory
- →Support cloud migration projects ensuring data security, quality, and integrity
- →Collaborate with business and data science teams for AI model deployment in cloud environments
- →Ensure data quality and consistency through data profiling and monitoring techniques
- →Design and maintain Data Lake and Data Warehouse solutions with data governance best practices
- →Document processes, architectures, and data engineering best practices
Requirements
- ✓Solid experience with Azure Data Services and Databricks
- ✓Proficiency in Python and SQL for data manipulation and transformation
- ✓Knowledge of CI/CD for data environments
- ✓Experience in data modeling (conceptual, logical, and physical)
- ✓Experience with cloud migration projects
- ✓Experience with ETL/ELT and large-volume data integration
- ✓Experience with AI model deployment and tools like Azure ML
- ✓Knowledge of data-oriented architectures and data governance best practices
- ✓Participation in Master Data Management (MDM) initiatives
- ✓Familiarity with Databricks MLflow for model management
- ✓Experience with Data Lake and Data Warehouse environments
Benefits
- ★Health and dental insurance plans
- ★Food and meal vouchers
- ★Childcare assistance
- ★Extended parental leave
- ★Gym partnerships and wellness programs via Wellhub and TotalPass
- ★Profit and results sharing (PLR)
- ★Life insurance
- ★Continuous learning platform (CI&T University)
- ★Discount club
- ★Online platform for physical, mental health and wellbeing
- ★Parenting courses
- ★Online course platform partnerships
- ★Language learning platform
