Loading...
Mid LevelFull-timeTypeScriptPythonRustReactRailsAIML
Salary Range
$102,000 - $138,000 /year
This salary is estimated based on similar roles. The actual salary may vary.
About the Role
Fieldguide is seeking an AI Engineer, Quality to build evaluation infrastructure for their AI agents. This role focuses on ensuring AI agents perform reliably at enterprise scale by developing unified platforms, automated pipelines, and production feedback loops.
Key Responsibilities
- →Design and build a unified evaluation platform for agentic systems and audit workflows
- →Build observability systems to surface agent behavior and trace execution
- →Own the evaluation infrastructure stack including integration with LangSmith and LangGraph
- →Build automated pipelines to evaluate new models against critical workflows
- →Design evaluation harnesses for complex agentic systems
- →Implement comparison frameworks to measure effectiveness and consistency
- →Define and document evaluation standards and best practices
- →Advocate for evaluation-driven development
Requirements
- ✓Multiple years of experience shipping production software in complex systems
- ✓Experience with TypeScript, React, Python, and Postgres
- ✓Built and deployed LLM-powered features serving production traffic
- ✓Implemented evaluation frameworks for model outputs and agent behaviors
- ✓Designed observability or tracing infrastructure for AI/ML systems
- ✓Experience with vector databases, embedding models, and RAG architectures
- ✓Experience with evaluation platforms like LangSmith or Langfuse
- ✓Comfort operating in ambiguity and taking responsibility for outcomes
- ✓Deep empathy for professional-grade, mission-critical software
Benefits
- ★Opportunity to build out the future of business trust
- ★Remote-first company culture
- ★Work with top-tier investors and accounting/consulting firms
- ★Inclusive, driven, humble, and supportive team environment
Free to sign up. Takes 30 seconds.
