Jobs → Engineering → Python/Java Software Engineer

About the Role We are seeking an experienced Python/Java Software Engineer passionate about AI, LLMs, and AI-powered software engineering tools. This role focuses on building intelligent evaluation frameworks, automating software development workflows, and benchmarking AI coding agents. Key Responsibilities • Develop automated evaluation frameworks for AI-powered software development tools • Build reproducible benchmarks using real-world engineering artifacts, including merged pull requests • Evaluate AI models and coding agents for quality, productivity, accuracy, cost, and latency • Analyze test results, performance variations, and failure patterns to improve reliability • Validate AI evaluation metrics against human judgment and document technical findings Requirements • Proficiency in Python and Java/JavaScript • Experience with LLMs, Claude Code Ecosystem, OpenCode, and RAG Pipelines • Knowledge of AI Agent Evaluation & Benchmarking • Familiarity with Git, Docker, APIs, and CI/CD Pipelines • Experience with Test Automation, Reproducible Environments, and Performance Analysis • Must be legally authorized to work in the United States Benefits • Medical, dental, and vision insurance (eligibility varies based on employment status and assignment)