Apache Spark & Scala Engineer
Market Cloud LtdUnited KingdomMIDPosted 29m ago
ScalaApache SparkSpark SQLDistributed ComputingBig DataETLELTSQLParquetAvroJSONORCAWSAzureGCP
About the Role
We are looking for an experienced Apache Spark & Scala Engineer / Developer to join our client's technology team.
Key Responsibilities
• Design and implement scalable data solutions using Apache Spark and Scala
• Build and optimize large-scale ETL/ELT data pipelines
• Work with distributed computing and Big Data technologies
• Develop solutions using SQL, databases, and data processing frameworks
• Utilize data formats such as Parquet, Avro, JSON, and ORC
Requirements
• 8–12 years of experience in relevant technologies
• Strong hands-on experience with Scala programming
• Extensive experience with Apache Spark and Spark SQL
• Strong understanding of distributed computing and Big Data technologies
• Experience building and optimising large-scale ETL/ELT data pipelines
• Good knowledge of SQL, databases, and data processing frameworks
• Experience with Parquet, Avro, JSON, and ORC data formats
• Familiarity with cloud platforms (AWS/Azure/GCP)
Good to Have
• Experience with Apache Kafka or other streaming technologies
• Databricks and cloud-based Spark platforms
• Docker, Kubernetes, and CI/CD
• Financial services or enterprise data environment experience