Data Engineer
Prathap KumarUnited StatesMIDPosted 35m ago
DatabricksPySparkSQLSnowflakeUnity CatalogETL/ELT PipelinesApache AirflowData IngestionData QualityData MigrationPipeline MonitoringPerformance Tuning
About the Role
We are looking for an experienced Data Engineer to join our team and support data engineering initiatives for Disney. The ideal candidate should have strong hands-on experience with Databricks, PySpark, SQL, Snowflake, and Unity Catalog.
Key Responsibilities
• Build and maintain ETL/ELT pipelines using Databricks, PySpark, SQL, and Unity Catalog
• Migrate existing Snowflake SQL scripts and transformations into Databricks Unity Catalog
• Develop data ingestion processes for marketing vendor data, including parsing, normalization, and data validation
• Implement data quality checks, monitoring alerts, and issue-tracking workflows
• Work with Databricks, Snowflake, and Airflow for pipeline monitoring and reliability
• Support SLA monitoring, incident troubleshooting, and logging improvements
• Collaborate with analytics and marketing teams to understand data requirements
• Perform performance tuning and improve pipeline reliability
• Participate in code reviews, documentation, and engineering best practices
Requirements
• Strong hands-on experience with Databricks, PySpark, SQL, Snowflake, and Unity Catalog
• Experience with ETL/ELT Pipelines
• Knowledge of Apache Airflow
• Understanding of Data Ingestion & Data Quality
• Experience with Data Migration
• Skills in Pipeline Monitoring & Performance Tuning