Jobs → QA & Testing → GenAI QA Engineer

GenAI QA Engineer

Shikha Singh K.RemoteMIDPosted 2h ago
GenAILLM Application TestingRAG architecturepipeline validationLLM evaluation frameworksRagasTruLensDeepEvalLLM evaluation metricsFaithfulnessAnswer RelevanceContext Precision/RecallHallucination RatePrompt InjectionJailbreakAI Safety TestingPythonTypeScriptAPI TestingPostmanBrunocURLAutomationPyTestPlaywrightCypressGCP servicesVertex AIBigQueryCloud StorageCloud Logging
About the Role We are looking for a GenAI QA Engineer with expertise in RAG and GCP to join our team. This role focuses on testing GenAI and LLM applications, ensuring quality and reliability. Key Responsibilities • Design and implement test cases for GenAI and LLM applications • Validate RAG architecture and pipelines • Evaluate LLM performance using frameworks like Ragas, TruLens, or DeepEval • Test for Prompt Injection, Jailbreak, and AI Safety • Automate testing processes using Python or TypeScript • Perform API testing using tools like Postman, Bruno, or cURL • Utilize GCP services such as Vertex AI, BigQuery, Cloud Storage, and Cloud Logging Requirements • 4–8 years of experience in QA Automation or Software Testing • Hands-on experience in GenAI / LLM Application Testing • Strong understanding of RAG architecture and pipeline validation • Experience with LLM evaluation frameworks: Ragas, TruLens or DeepEval • Knowledge of LLM evaluation metrics – Faithfulness, Answer Relevance, Context Precision/Recall and Hallucination Rate • Experience in Prompt Injection, Jailbreak and AI Safety Testing • Strong Python or TypeScript programming skills • API Testing using Postman, Bruno or cURL • Automation experience with PyTest, Playwright or Cypress • Working knowledge of GCP services – Vertex AI, BigQuery, Cloud Storage and Cloud Logging

Similar projects

UiPath Test Automation Engineer
Anjani K. · Python, Bash scripting, UiPath
QA Lead
Nithish Jayaprakash · Java, Selenium, RestAssured
QA SDET
MOHAMMAD R · Generative AI, Quality Engineering, SCM Supply Chain Management
Early · 0 applied1 of 6,102 projects

Apply with AI cover letter

AI writes a personalized application in 19 seconds. Just upload your resume.

No credit card · First application free
© 2026 Freelanly · Terms · Privacy