Python Data Engineer
Challa Rupasri ReddySENIORPosted 12d ago
PythonPySparkSQLOracleETLData PipelinesData ModelingSpark DataFramesKafkaAirflowSnowflakeData LakesGenAIML
About the Role
We are looking for a Python Data Engineer to join our team. This role focuses on designing, developing, and maintaining data pipelines using Python and PySpark.
Key Responsibilities
• Design, develop, and maintain data pipelines using Python and PySpark
• Build and optimize ETL/ELT processes for large datasets
• Perform data transformations, validation, and reconciliation using SQL and Oracle
• Develop data models and improve data processing performance
• Optimize Spark jobs, joins, partitioning, and SQL queries
• Implement data-quality checks and troubleshoot production issues
• Collaborate with database, application, QA, and DevOps teams
Requirements
• Strong Python and PySpark experience
• Advanced SQL and Oracle database knowledge
• ETL/ELT, Data Pipelines, and Data Modeling
• Spark DataFrames, transformations, joins, caching, and optimization
• Production troubleshooting and data-quality management
• Banking or Financial Services experience preferred
Nice to Have
• Databricks, AWS/Azure, Kafka, Airflow, Snowflake, Data Lakes, GenAI, and ML