Data Engineer with CDP, GCP, Cloud Spanner
Pankaj kumarUnited StatesSENIORPosted 12d ago
CDPGCPCloud SpannerBigQueryGraphQLMCPPub/SubDataflowBeamSpannerPySpark
About the Role
We are looking for a Data Engineer with expertise in CDP, GCP, and Cloud Spanner to build and operate real-time pipelines for a Customer Data Platform. This role involves migrating and reusing proven ETL from the Azure estate to Google Cloud.
Key Responsibilities
• Build and run NRT ingestion with Pub/Sub and Dataflow (Beam) into Spanner and BigQuery
• Implement Spanner loads and change streams for current-state customer profiles
• Implement reusable CDC, SCD Type 2, and late-arriving-data patterns in Beam, SQL, and PySpark
• Build clickstream ingestion at scale
Requirements
• Experience with CDP, GCP, Cloud Spanner, BigQuery, and GraphQL
• Strong understanding of data modeling and real-time pipeline engineering
• Experience with PySpark, Beam, and SQL
• Background in FAANG companies preferred
Benefits