Jobs → QA & Testing → GenAI Automation Tester

GenAI Automation Tester

Shweta kaleRemoteMIDPosted 7d ago
PlaywrightTypeScriptLLM TestingUI Automation TestingAPI Automation TestingE2E TestingRAG TestingHallucination DetectionPrompt TestingPrompt Injection TestingMCPAgentic AI TestingCI/CDJenkinsAzure DevOpsGitHub ActionsDeepEvalPromptFooLangSmithRAGASLLM-as-a-JudgeLangChainLangGraph
About the Role We are looking for a GenAI Automation Tester to join our team. This role focuses on testing large language models and AI systems. Key Responsibilities • Perform UI and API automation testing for GenAI applications • Conduct end-to-end testing of AI systems • Implement RAG testing and hallucination detection • Perform prompt testing and prompt injection testing • Test MCP and agentic AI systems • Set up and maintain CI/CD pipelines Requirements • 4–8 years of experience in automation testing • Strong knowledge of Playwright and TypeScript • Experience with LLM testing and RAG testing • Understanding of hallucination detection and prompt testing • Familiarity with CI/CD tools like Jenkins, Azure DevOps, and GitHub Actions • Knowledge of DeepEval, PromptFoo, LangSmith, RAGAS, and LLM-as-a-Judge is a plus • Experience with LangChain and LangGraph is a plus

Similar projects

QA Engineer
Shania Salsabila · QA, Software Testing, SDET
AEM QA Engineer
Archanaa Srivastav · AEM, Adobe Experience Manager, AEM 6.x
Salesforce CPQ QA Engineer
Archanaa Srivastav · Salesforce Testing, Salesforce CPQ, Manual Testing
Early · 0 applied1 of 7,999 projects

Apply with AI cover letter

AI writes a personalized application in 19 seconds. Just upload your resume.

No credit card · First application free
© 2026 Freelanly · Terms · Privacy