Python/PySpark Data Engineer
Shaik SafreenUnited StatesMIDPosted 1h ago
PythonPySparkApache SparkOracleSQLREST APIETLELTData WarehousingData ModelingMetadata ManagementData LineageData GovernanceKafkaGoogle Cloud PlatformAb InitioServiceNow
About the Role
We are looking for a highly skilled Python/PySpark Data Engineer to design and build scalable data ingestion and integration pipelines across enterprise data sources and metadata platforms.
Key Responsibilities
• Design and build scalable data ingestion and integration pipelines
• Work with enterprise data sources and metadata platforms
• Implement ETL/ELT processes and data warehousing solutions
• Manage metadata, data lineage, and data governance
• Integrate REST APIs and batch data processing
Requirements
• Strong experience with Python, PySpark, and Apache Spark
• Knowledge of Oracle/SQL and relational databases
• Experience with REST API integration
• Understanding of ETL/ELT, data warehousing, and data modeling
• Familiarity with metadata management, data lineage, and data governance
• Experience with batch data ingestion and integration pipelines
• Experience with enterprise data frameworks