Staff Software Engineer, Inference at Anthropic

Anthropic

Dublin, IE·13h ago

LeadFull-timePythonRustAWSGCPKubernetesMachine LearningGitAIDevOps

Salary Range

$110,160 - $149,040 /year

Estimated

This salary is estimated based on similar roles. The actual salary may vary.

About the Role

We are looking for a Staff Software Engineer to join our Inference team at Anthropic. This role involves building and maintaining critical systems that serve Claude to millions of users worldwide, focusing on performance optimization, distributed systems, and large-scale service orchestration.

Key Responsibilities

→Design and implement intelligent routing algorithms to optimize request distribution
→Autoscale compute fleet to match demand across production and research workloads
→Build deployment pipelines for releasing new models to millions of users
→Integrate new AI accelerator platforms to maintain hardware-agnostic advantage
→Contribute to new inference features like structured sampling and prompt caching
→Support inference for new model architectures
→Analyze observability data to tune performance based on production workloads
→Manage multi-region deployments for global customers

Requirements

✓Significant software engineering experience, particularly with distributed systems
✓Familiarity with performance optimization, distributed systems, and large-scale service orchestration
✓Experience with LLM inference optimization, batching strategies, and multi-accelerator deployments
✓Strong understanding of Kubernetes and cloud infrastructure (AWS, GCP)
✓Proficiency in Python or Rust
✓Results-oriented with a bias towards flexibility and impact
✓Willingness to learn about machine learning systems and infrastructure
✓Ability to thrive in environments where technical excellence drives both business results and research breakthroughs
✓Commitment to considering the societal impacts of your work