AI Benchmark Engineer - Native Language Specialist | Czech at Lilt Production

Lilt Production

Czech Republic (Remote)·13h ago

Mid LevelContractPythonAWSAI

Salary Range

$102,000 - $138,000 /year

Estimated

This salary is estimated based on similar roles. The actual salary may vary.

About the Role

We are seeking experienced native-speaking software engineers to design, build, and validate rigorous evaluation suites for Terminal-Bench tasks. These tasks will test the limits of large language models on multilingual software challenges.

Key Responsibilities

→Task Engineering: Evaluating Coding Agents
→Asset Creation: Build realistic task environments using datasets and files in your native language
→Prompting & Translation: Finding failure points where AI does not work, in your native language
→Implementation & Verification: Support the development of robust solutions and write verifier scripts
→Calibration & Execution: Analyze execution logs and calibrate task difficulty
→Quality Assurance: Participate in a 4-layer human quality control process

Requirements

✓5+ years of industry experience in software engineering
✓Proven track record at leading technology companies and/or graduation from top-tier engineering universities
✓Native or near-native fluency in one of the target languages with a deep understanding of its grammar, register, and phrasing rules
✓High English proficiency
✓Strong proficiency in Python, standard shell scripting, and data processing
✓Extensive experience with Terminal/CLI-based development workflows and a working familiarity with coding agents
✓Deep technical understanding of multilingual text processing pitfalls, including encoding/decoding robustness, locale-dependent conventions, and text I/O, toolchain interoperability, and safe string operations
✓(For specific languages) Bidirectional/RTL handling, font fallbacks, and rendering/typography in UI or artifacts