Data Engineer
Vijaya laxmi ARemoteSENIORPosted 7d ago
PythonDataFramesPandasPySparkETLELTdata processingdata integrationdata modelingdata governancedata securitydata orchestration
About the Role
We are seeking a highly skilled Data Engineer to design, build, and maintain scalable data pipelines and ingestion frameworks. This role involves transforming raw data into reliable, high-quality datasets that support analytics, reporting, and business decision-making.
Key Responsibilities
• Design, develop, and maintain robust data ingestion and ETL/ELT pipelines
• Build scalable data processing solutions using Python, Pandas, and PySpark
• Develop and optimize data workflows for batch and real-time processing
• Integrate data from multiple sources including APIs, databases, files, and cloud platforms
• Perform data cleansing, transformation, validation, and enrichment activities
• Create and manage data models, schemas, and metadata frameworks
• Ensure data quality, integrity, governance, and security standards are maintained
• Monitor, troubleshoot, and optimize data pipeline performance
• Collaborate with Data Analysts, Data Scientists, and Business Teams to deliver reliable data solutions
• Automate data ingestion and processing workflows using orchestration tools
• Document data architecture, pipelines, and technical solutions
Requirements
• 8+ years of experience in data engineering
• Strong expertise in Python, DataFrames (Pandas/PySpark), and data processing libraries
• Experience in ETL/ELT development and large-scale data integration
• Knowledge of data modeling, schemas, and metadata frameworks
• Understanding of data quality, integrity, governance, and security standards
• Experience with data orchestration tools
• Ability to collaborate with cross-functional teams