Loading...
SeniorFull-timeRustData ScienceAI
Salary Range
$132,600 - $179,400 /year
This salary is estimated based on similar roles. The actual salary may vary.
About the Role
The AI Linguist plays a key role in shaping the quality of LinkedIn's AI systems across various use cases, including relevance, ranking, rationale quality, and emerging multi-step and agentic capabilities. This role involves designing annotation tasks and rubrics, producing high-quality ground truth through hands-on annotation, and building frameworks that make AI systems trainable, measurable, and continuously improvable.
Key Responsibilities
- →Partner cross-functionally with Engineering, Product, Data Science, domain SMEs, Trust/Legal, TPM, and vendor ops to align on quality goals, tradeoffs, and delivery plans
- →Define measurable quality criteria for ambiguous behaviors and ensure consistency across markets
- →Design and run repeatable evaluation systems, including multi-step/agentic behavior evaluation
- →Build scalable, high-quality annotation/evaluation pipelines, including hands-on execution of annotation tasks
- →Lead vendor and internal workforce execution at scale, owning training and onboarding, calibration sessions, periodic reviews/audits
- →Establish and enforce quality governance (agreement targets, drift/bias checks, defect taxonomy)
- →Leverage AI tools to scale work while maintaining reliability controls
- →Run method/workflow experiments; document results and drive decisions based on evidence
- →Perform error analysis and drive iteration cycles with partners; translate findings into actionable changes
- →Define platform/tool requirements for human judgment workflows; partner through build/test/deploy and adoption
- →Publish reusable best practices and standards; mentor junior Linguists and conduct design/analysis reviews across initiatives
Requirements
- ✓Experience in designing annotation tasks and rubrics
- ✓Strong understanding of AI systems, models, and agents
- ✓Ability to work with cross-functional teams and stakeholders
- ✓Experience with LLM-assisted prompting, hybrid labeling, and evaluation
- ✓Knowledge of automated checks, regression testing, and continuous monitoring
- ✓Ability to lead vendor and internal workforce execution at scale
- ✓Strong analytical and problem-solving skills
- ✓Experience with quality governance and inter-annotator agreement
- ✓Ability to leverage AI tools to scale work while maintaining reliability controls
- ✓Experience with method/workflow experiments and documentation
- ✓Ability to perform error analysis and drive iteration cycles
- ✓Experience with platform/tool requirements for human judgment workflows
- ✓Ability to publish reusable best practices and standards
- ✓Experience mentoring junior Linguists and conducting design/analysis reviews
Free to sign up. Takes 30 seconds.
