Loading...
[2026] Data Scientist, Foundation AI - PhD Early Career at Roblox
RobloxSan Mateo, CA, United States·12h ago
Mid LevelFull-timePythonAWSMachine LearningData ScienceGitAI
Salary Range
$80,750 - $109,250 /year
This salary is estimated based on similar roles. The actual salary may vary.
About the Role
We are looking for a Data Scientist to join our Data Science & Analytics organization at Roblox. This role focuses on evaluation and optimization for user-facing GenAI systems, including text, image, video, 3D, and 4D. You will define how we measure safety, responsibility, quality, and efficiency in these systems.
Key Responsibilities
- →Develop Evaluation Frameworks: Design and operationalize rigorous evaluation systems for GenAI features, including eval experiment design, dataset design, label reliability analysis, and implementing LLM-as-a-judge methods.
- →Run Rigorous Experiments: Conduct online experiments (A/B tests) and causal inference to quantify the impact of GenAI features, identifying opportunities and ensuring statistical rigor.
- →Define Success Metrics: Partner with cross-functional teams to define leading/lagging indicators for GenAI feature user satisfaction, business success, and safety.
- →Build Automated Systems: Research and apply state-of-the-art methodologies to build reproducible evaluation tooling that lift rigor and efficiency across the company.
- →Conduct Applied Research: Maintain an active pulse on the intersection of Gen AI and Data Science, innovating on methodology and techniques to solve unique business challenges.
Requirements
- ✓Possess or pursuing a PhD or equivalent in Statistics, Economics, Computer Science, Applied Math, Physics, Engineering, or a related quantitative field.
- ✓Technical Proficiency: Strong proficiency in SQL (Hive/Spark) for manipulating large datasets and scripting languages (Python or R) for analysis and modeling.
- ✓Experimentation and Causal Inference: A solid grounding in experimentation, causal inference, and statistical analysis, including test design and metric design for feature impact.
- ✓Problem Solving: A demonstrated track record of framing ambiguous problems, designing analytical approaches, and solving open-ended data science problems that drive business impact.
- ✓Learning Agility: Ability to effectively and responsibly use AI tools to enhance productivity and a passion for continuously improving methods in a fast-evolving field.
- ✓GenAI Familiarity: Familiarity with GenAI models and safety/quality evaluation methods. Expertise in the model training lifecycle is a plus.
- ✓Applied Research Background: A track record of applied research or publications in relevant technical fields is highly valued.
