Loading...
SeniorFull-timeMicroservicesCI/CDGitAIAPI
Salary Range
$26,520 - $35,880 /year
This salary is estimated based on similar roles. The actual salary may vary.
About the Role
We are seeking a highly skilled Test Automation Engineer to join the Agentic AI Squad within our AI Platform organization. This role focuses on designing, implementing, and maintaining automated test frameworks for complex distributed systems, with a particular emphasis on AI/LLM-powered capabilities.
Key Responsibilities
- →Design, implement, and maintain scalable automated test suites for API, integration, end-to-end, and workflow-level testing of agentic AI systems.
- →Build automation frameworks that validate correctness, robustness, safety, and determinism in LLM-driven behaviors and multi-agent orchestration.
- →Develop synthetic datasets, fixtures, and mocks to replicate complex agentic scenarios and edge cases.
- →Implement automated evaluation pipelines for LLM outputs, including functional tests, regression checks, safety tests, and model-behavior verification.
- →Collaborate with AI Engineers to integrate model evaluation metrics into CI/CD systems.
- →Validate prompt templates, agent tools, retrieval workflows, and chain-of-thought instrumentation.
- →Work with backend and platform teams to ensure test coverage across microservices, feature flags, workflow orchestrators, and event-driven systems.
- →Contribute to testable system design and ensure new features include clear acceptance criteria and automation strategies.
- →Integrate automated testing pipelines into GitHub Actions, ensuring fast feedback cycles and continuous verification.
- →Implement quality gates, flakiness detection, and test result analytics to improve reliability.
- →Participate in release readiness reviews and ensure that automated checks provide full coverage for functional and non-functional requirements.
- →Understand user feedback and reproduce bugs that have been submitted to the team.
- →Perform root cause analysis to enable AI Engineers to work on bug fixes and provide guidance on acceptance criteria.
