Loading...
Entry LevelFull-timePythonAIMLData
Salary Range
$17,850 - $24,150 /year
This salary is estimated based on similar roles. The actual salary may vary.
About the Role
We are seeking a Data Engineer to join our team and play a key role in data migration and optimization for advanced AI and ML projects. This position involves designing and implementing data pipelines to ensure seamless data flow and quality.
Key Responsibilities
- →Design, develop, and deploy Python-based ETL/ELT pipelines to migrate data from on-premises MS SQL Server into the Databricks instance
- →Ensure efficient ingestion of historical parquet datasets into Databricks
- →Implement validation, reconciliation, and quality assurance checks to ensure accuracy and completeness of migrated data
- →Handle schema mapping, field transformations, and metadata enrichment to standardize datasets
- →Tune pipelines for speed and efficiency, leveraging Databricks capabilities such as Delta Lake
- →Work closely with AI engineers, data scientists, and business stakeholders to define data access patterns
- →Partner with infrastructure teams to ensure secure connection between legacy systems and Databricks
- →Maintain technical documentation for all data pipelines
- →Adhere to data governance, compliance, and security best practices throughout the migration process
Requirements
- ✓Experience with Python and ETL/ELT processes
- ✓Strong understanding of MS SQL Server and Databricks
- ✓Knowledge of data quality and governance principles
- ✓Ability to work with large datasets and optimize performance
- ✓Experience with Delta Lake and other Databricks capabilities
- ✓Strong collaboration and communication skills
Free to sign up. Takes 30 seconds.
About Inetum2
Inetum2 is a technology consulting and services company specializing in digital transformation and IT solutions. Operating within the IT services industry, the company focuses on leveraging cutting-edge technologies to help businesses adapt and thrive in a rapidly evolving digital landscape. While the name suggests a connection to the larger Inetum group, Inetum2 operates as a distinct entity, likely founded to address specific market needs or regional demands, though precise founding details are not explicitly stated. The company’s approach combines technical expertise with strategic advisory to deliver tailored solutions that align with client objectives. Inetum2 offers a range of products and services, including software development, system integration, cloud computing, and process optimization. Key features of their offerings include agile methodologies, scalable solutions, and a strong emphasis on cybersecurity and data privacy. Their target audience primarily consists of mid-to-large enterprises across sectors such as finance, healthcare, and manufacturing, which require robust IT infrastructure and innovative digital strategies to enhance operational efficiency and competitive advantage. The company’s solutions are designed to integrate seamlessly with existing systems while future-proofing clients’ technological investments. Notable achievements for Inetum2 may include successful client transformations and partnerships with industry leaders, though specific accolades are not detailed. The company’s market position is strengthened by its ability to provide end-to-end digital services, from initial consultation to ongoing support. Its unique value proposition lies in its blend of technical proficiency and business acumen, ensuring that solutions not only meet technical requirements but also drive tangible business outcomes. This dual focus positions Inetum2 as a trusted partner for organizations navigating complex digital challenges.
