Senior QA Engineer
Srikanth SamudralaUnited StatesSENIORPosted 44m ago
PythonAPI AutomationAI Evaluation FrameworksLangChainLangGraphAWS BedrockRAGASDeepEvalLangSmithSQLAgile
About the Role
We are seeking a hands-on Senior QA Engineer with strong experience testing Agentic AI, LLM, and RAG-based applications.
Key Responsibilities
• Test AI agent reasoning, decision-making, tool calling, and multi-agent orchestration
• Validate LLM accuracy, hallucinations, response consistency, and contextual relevance
• Perform end-to-end testing of RAG pipelines, retrieval mechanisms, and AI workflows
• Develop Python-based test automation and API testing frameworks
• Build automated AI evaluation and regression testing solutions
• Validate prompts, edge cases, failure scenarios, and AI guardrails
• Hands-on exposure to LangChain, LangGraph, and AWS Bedrock
• Experience with RAGAS, DeepEval, and LangSmith
Requirements
• 8+ years QA / Test Automation
• Hands-on Agentic AI / LLM / RAG testing
• Python & API Automation
• AI Evaluation Frameworks
• LangChain / LangGraph
• Strong SQL, debugging, and Agile experience