AI Engineer
Marzia Rahman SumaitaUnited StatesMIDPosted 11h ago
LangGraphLLM EvalsPythonNode.jsJavaScriptTypeScriptAWS BedrockClaudeAWSLLM observabilityArizeOpenTelemetry
About the Role
We are looking for an AI Engineer to build an internal AI developer platform where agents are treated like real software: stateful, tested and measured.
Key Responsibilities
• Build and debug LangGraph agent workflows (Python) with human-in-the-loop steps
• Extend the eval harness and run A/B tests between agent and prompt variants
• Improve prompts through measured, regression-tested changes
• Extend the collaborative human-agent UI (Node.js)
Requirements
• Hands-on LangGraph: state graphs, checkpointing, interrupts, tool-calling agents
• LLM evals: golden datasets, LLM-as-judge, A/B comparisons on quality, cost and latency
• Strong prompt engineering, including structured outputs
• Python; Claude on AWS Bedrock a plus
Nice to have
• JavaScript/TypeScript front end (Tiptap or Y.js a plus)
• LLM observability (Arize, OpenTelemetry)
• Node.js and AWS