GenAI Automation Tester
Shweta kaleRemoteMIDPosted 7d ago
PlaywrightTypeScriptLLM TestingUI Automation TestingAPI Automation TestingE2E TestingRAG TestingHallucination DetectionPrompt TestingPrompt Injection TestingMCPAgentic AI TestingCI/CDJenkinsAzure DevOpsGitHub ActionsDeepEvalPromptFooLangSmithRAGASLLM-as-a-JudgeLangChainLangGraph
About the Role
We are looking for a GenAI Automation Tester to join our team. This role focuses on testing large language models and AI systems.
Key Responsibilities
• Perform UI and API automation testing for GenAI applications
• Conduct end-to-end testing of AI systems
• Implement RAG testing and hallucination detection
• Perform prompt testing and prompt injection testing
• Test MCP and agentic AI systems
• Set up and maintain CI/CD pipelines
Requirements
• 4–8 years of experience in automation testing
• Strong knowledge of Playwright and TypeScript
• Experience with LLM testing and RAG testing
• Understanding of hallucination detection and prompt testing
• Familiarity with CI/CD tools like Jenkins, Azure DevOps, and GitHub Actions
• Knowledge of DeepEval, PromptFoo, LangSmith, RAGAS, and LLM-as-a-Judge is a plus
• Experience with LangChain and LangGraph is a plus