Loading...
SeniorFull-timeAzureMachine Learning
Salary Range
$95,472 - $129,168 /year
This salary is estimated based on similar roles. The actual salary may vary.
About the Role
We are seeking a Data Engineer to design, develop, and maintain scalable data pipelines on Databricks and cloud platforms. This role involves integrating diverse data sources, ensuring high-quality data processing, and supporting analytics, reporting, and machine learning workloads.
Key Responsibilities
- →Develop and maintain ETL pipelines for centralized data storage systems
- →Integrate data from databases, APIs, log files, streaming platforms, and external providers
- →Develop data transformation routines to clean, normalize, and aggregate data
- →Apply data processing techniques to handle complex or inconsistent datasets
- →Contribute to frameworks and best practices for code development and deployment
- →Implement data governance in alignment with company standards
- →Partner with analytics and product leaders to design and operationalize pipelines
- →Collaborate with infrastructure leaders to advance cloud-based data platforms
- →Explore new tools and techniques leveraging Azure, Databricks, or related platforms
- →Monitor data pipelines to detect and resolve issues promptly
- →Develop monitoring tools, alerts, and automated error-handling mechanisms
- →Analyse business requirements and identify data extraction requirements
- →Attend requirement grooming and refinement sessions with users
- →Develop and maintain ETL pipelines for ingestion, transformation, validation, and loading
- →Optimise performance and batch scheduling
- →Develop dashboards, reports, scorecards, and data visualizations
- →Perform SIT, data validation, data profiling and confirm data accuracy
- →Validate completeness and consistency of ETL Loads
- →Support UAT and production implementation
Requirements
- ✓Experience with Databricks and cloud platforms
- ✓Strong knowledge of ETL processes and data pipelines
- ✓Proficiency in data transformation and normalization techniques
- ✓Understanding of data governance and best practices
- ✓Ability to integrate diverse data sources
- ✓Experience with Azure and related platforms
- ✓Skills in developing monitoring tools and automated error-handling mechanisms
- ✓Ability to analyze business requirements and identify data extraction needs
- ✓Experience with data validation, profiling, and accuracy confirmation
- ✓Support for UAT and production implementation
Free to sign up. Takes 30 seconds.
About NCS3
NCS3 is a forward-thinking technology company specializing in innovative solutions for the digital transformation landscape. Operating at the intersection of cloud computing, data analytics, and cybersecurity, NCS3 provides cutting-edge services to help organizations navigate the complexities of modern IT infrastructure. While the name suggests a focus on core computing systems, the company has established itself as a trusted partner for businesses seeking scalable and secure technological frameworks. Founded in the era of rapid digital evolution, NCS3 leverages its expertise to address the evolving needs of enterprises across various sectors. The company offers a comprehensive suite of products and services, including cloud migration strategies, advanced data management tools, and robust security protocols tailored to protect sensitive information. Key features of NCS3’s offerings include seamless integration with existing systems, customizable solutions to meet specific business requirements, and a commitment to compliance with industry standards. Their target audience primarily consists of mid-to-large enterprises in finance, healthcare, and manufacturing, where reliability and data integrity are paramount. By focusing on practical, results-driven approaches, NCS3 ensures its clients achieve operational efficiency and competitive advantage. NCS3 has distinguished itself through notable achievements, such as securing partnerships with global technology leaders and receiving recognition for its innovative cybersecurity frameworks. The company’s market position is strengthened by its ability to deliver measurable outcomes, combining technical expertise with a deep understanding of client challenges. What sets NCS3 apart is its client-centric philosophy, which emphasizes long-term collaboration and continuous improvement, ensuring that solutions evolve alongside technological advancements and business needs.
