Loading...
AI Benchmark Engineer - Native Language Specialist | Serbian at Lilt Production
Lilt ProductionSerbia (Remote)·9h ago
Mid LevelContractPythonAWSAI
Salary Range
$102,000 - $138,000 /year
This salary is estimated based on similar roles. The actual salary may vary.
About the Role
We are seeking experienced native-speaking software engineers to design, build, and validate benchmarks for testing large language models on multilingual software challenges. This role involves creating high-quality tasks that test a model's ability to handle multilingual environments without relying on English translation.
Key Responsibilities
- →Task Engineering: Evaluating Coding Agents
- →Asset Creation: Build realistic task environments using datasets and files in your native language
- →Prompting & Translation: Finding failure points where AI does not work in your native language
- →Implementation & Verification: Support development of robust solutions and write verifier scripts
- →Calibration & Execution: Analyze execution logs and calibrate task difficulty
- →Quality Assurance: Participate in a 4-layer human quality control process
Requirements
- ✓5+ years of industry experience in software engineering
- ✓Proven track record at leading technology companies and/or graduation from top-tier engineering universities
- ✓Native or near-native fluency with deep understanding of grammar, register, and phrasing rules
- ✓High English proficiency
- ✓Strong proficiency in Python, standard shell scripting, and data processing
- ✓Extensive experience with Terminal/CLI-based development workflows and coding agents
- ✓Deep technical understanding of multilingual text processing pitfalls including encoding/decoding robustness, Unicode normalization, locale-dependent conventions, and text I/O operations
Free to sign up. Takes 30 seconds.
