Data Scientist – GenAI / AI Agent Evaluation
Pinakin KanthaRemoteMIDPosted 27m ago
PythonPandasScikit-learnSQLData ScienceMachine LearningPredictive ModelingGenAILLM ApplicationsAI Agent EvaluationNLPText AnalyticsSentimentIntent ClassificationDatabricksPySparkSpark SQLDelta TablesUnity CatalogFeature EngineeringBraintrustLangSmithArizeWeights & BiasesAWSBedrockLambdaS3CloudWatchREST APIsJSONSalesforceSOQLML Model EvaluationMLOpsProduction Deployment
About the Role
We are looking for a hands-on Data Scientist who can build, evaluate and optimize AI/LLM-powered applications, with strong expertise in Python, Databricks, AWS and production ML solutions.
Key Responsibilities
• Build evaluation frameworks, datasets, scorers and regression tests for AI agents and LLM applications.
• Develop predictive models for customer churn, customer health scoring, ticket escalation and case deflection.
• Analyze customer conversations, support tickets, emails and chats using NLP and GenAI.
• Build scalable data pipelines and feature engineering solutions using Databricks and Spark SQL.
• Integrate AI agent logs and traces with Salesforce, Braintrust, AWS and enterprise data platforms.
• Drive model quality measurement, performance monitoring and production handover.
Requirements
A hands-on Data Scientist with 5–9 years of overall experience, including 2+ years in Data Science, who combines strong Python + NLP + Databricks + AWS expertise with practical GenAI / AI Agent evaluation experience.