AI Benchmark Engineer - Native Language Specialist | Spanish at Lilt Production

Lilt Production

Spain (Remote)·19h ago

Mid LevelContractPythonAWSAI

Salary Range

$48,960 - $66,240 /year

Estimated

This salary is estimated based on similar roles. The actual salary may vary.

About the Role

We are seeking experienced native-speaking software engineers to design, build, and validate rigorous evaluation suites for Terminal-Bench tasks. This role focuses on creating high-quality benchmarks to test multilingual robustness in large language models.

Key Responsibilities

→Task Engineering: Evaluating Coding Agents
→Asset Creation: Build realistic task environments using datasets and files in your native language
→Prompting & Translation: Finding failure points where AI does not work in your native language
→Implementation & Verification: Support development of robust solutions and write verifier scripts
→Calibration & Execution: Analyze execution logs and calibrate task difficulty
→Quality Assurance: Participate in a 4-layer human quality control process

Requirements

✓5+ years of industry experience in software engineering
✓Proven track record at leading technology companies or top-tier engineering universities
✓Native or near-native fluency with deep understanding of grammar, register, and phrasing rules
✓High English proficiency
✓Strong proficiency in Python, standard shell scripting, and data processing
✓Extensive experience with Terminal/CLI-based development workflows
✓Deep technical understanding of multilingual text processing pitfalls