Data Platform Engineer
Praveena YadavRemoteSENIORPosted 12d ago
HBaseEMRSparkHadoopScalaS3AWSHBase/EMR migrationdata migrationdata validationcluster troubleshootingHBase 1.xHBase 2.xEMR 5.xEMR 7.xHBase SnapshotExportSnapshotHFilesCoprocessorsApache PhoenixHivePrestoIcebergKerberosKMSchecksumsrollbacksnapshot/restore
About the Role
We are looking for a Data Platform Engineer to support a production data platform modernisation and migration programme. This role focuses on migrating and modernising distributed data platforms while maintaining application compatibility, data integrity, performance, and operational stability.
Key Responsibilities
• Participate in migration planning, cluster-level troubleshooting, data validation, snapshot/restore activities, cutover, rollback, and post-migration verification
• Support a production data platform modernisation and migration programme
• Migrate and modernise distributed data platforms while maintaining application compatibility, data integrity, performance, and operational stability
Requirements
• 10+ years of experience in Data Engineering / Big Data Engineering
• Strong hands-on production experience with HBase
• Strong experience with Amazon EMR
• Strong experience with Apache Spark
• Strong understanding of Hadoop ecosystem
• Experience with Scala and/or Spark programming
• Experience with S3 and AWS-based data platforms
• Hands-on experience with HBase/EMR migration or modernization
• Experience with production data migration and cutover activities
• Experience validating migrated data through row/region counts, reconciliation, checksums, or equivalent validation techniques
• Experience troubleshooting distributed data platforms at the cluster level
• Strong understanding of dependencies between HBase, Spark, Hadoop, and application workloads
• Experience with production support and migration-related troubleshooting