GCP Data Engineer
Hemant KumarRemoteSENIORPosted 8h ago
BigQueryKafkaDataflowApache BeamSparkPySparkScalaPub/SubCloud ComposerAirflowSQLData ModelingFeature StoresVertex AIMLOpsDisaster RecoveryData ArchivalHIPAAPHI
About the Role
We are looking for an experienced GCP Data Engineer to join our team, focusing on platform development, machine learning, and analytics.
Key Responsibilities
• Design and implement data solutions using BigQuery, Kafka, and Dataflow
• Develop and maintain ML pipelines with Vertex AI
• Implement MLOps practices for data engineering
• Ensure data security and compliance with HIPAA/PHI standards
• Design disaster recovery solutions for data systems
Requirements
• 12+ years of experience in data engineering
• Advanced skills in BigQuery, Kafka, and Dataflow
• Experience with Spark (PySpark or Scala) and Pub/Sub
• Strong SQL and data modeling skills
• Knowledge of Cloud Composer and Airflow
• Experience with feature stores and MLOps tools
• Hands-on experience with HIPAA/PHI controls
• Understanding of data archival and retention practices