GCP Data Engineer
Arvind SubramanianUnited StatesSENIORPosted 7d ago
Google Cloud PlatformBigQueryCloud DataflowCloud Pub/SubCloud StorageCloud ComposerVertex AILookerAmazon S3AWS GlueAmazon RedshiftAmazon EMRAWS LambdaKinesisAthenaPythonJavaSQLPySparkApache SparkApache AirflowDockerKubernetesDimensional modelingStar/Snowflake schemaData lakesNoSQL/SQL database design
About the Role
We are looking for a GCP Data Engineer to design, build, and maintain scalable cloud data infrastructure, data pipelines (ETL/ELT), and data warehouses across Google Cloud Platform and Amazon Web Services.
Key Responsibilities
• Design and implement scalable cloud data infrastructure
• Build and maintain data pipelines (ETL/ELT)
• Develop and manage data warehouses
• Work with BigQuery, Cloud Dataflow, Cloud Pub/Sub, Cloud Storage, Cloud Composer, Vertex AI, and Looker on GCP
• Utilize Amazon S3, AWS Glue, Amazon Redshift, Amazon EMR, AWS Lambda, Kinesis, and Athena on AWS
• Develop in Python, Java, SQL, PySpark, Apache Spark, and Apache Airflow
• Implement Docker and Kubernetes for containerization
• Apply dimensional modeling (Star/Snowflake schema), data lakes, and NoSQL/SQL database design
Requirements
• Experience with Google Cloud Platform and Amazon Web Services
• Proficiency in BigQuery, Cloud Dataflow, Cloud Pub/Sub, Cloud Storage, Cloud Composer, Vertex AI, and Looker
• Knowledge of Amazon S3, AWS Glue, Amazon Redshift, Amazon EMR, AWS Lambda, Kinesis, and Athena
• Strong programming skills in Python, Java, SQL, PySpark, Apache Spark, and Apache Airflow
• Experience with Docker and Kubernetes
• Understanding of dimensional modeling, data lakes, and NoSQL/SQL database design
• Preferred certifications: Google Cloud Professional Data Engineer or AWS Certified Data Analytics – Specialty