Data Engineer
Kr. R.RemoteMIDPosted 13d ago
DatabricksSparkPySparkPythonScalaSQLSnowflakeAzure Cloud ServicesADFADLSAzure FunctionsKey VaultDevOpsLLM Fine-TuningRAGPrompt EngineeringAirflowGitHubGitHub ActionsCI/CDTerraformInfrastructure AutomationKafkaEvent-Driven ArchitectureCDCStreaming Integrations
About the Role
We are looking for a Data Engineer to join our team. This role involves working with various data engineering tools and cloud services to build scalable data solutions.
Key Responsibilities
• Work with Databricks, Spark, and SQL for data processing
• Utilize Azure Cloud Services including ADF, ADLS, and Azure Functions
• Implement AI/GenAI solutions including LLM Fine-Tuning and RAG
• Manage automation and DevOps using Airflow, GitHub, and Terraform
• Handle Kafka and event-driven architecture for data streaming
Requirements
• Strong experience with Databricks, Spark (PySpark), Python/Scala, and SQL
• Proficiency in Azure Cloud Services, ADF, ADLS, Azure Functions, and Key Vault
• Knowledge of AI/GenAI concepts including LLM Fine-Tuning, RAG, and Prompt Engineering
• Experience with automation and DevOps tools like Airflow, GitHub, CI/CD, and Terraform
• Understanding of Kafka, Event-Driven Architecture, and CDC/Streaming Integrations