Loading...
Mid LevelFull-timePythonC++KubernetesMachine LearningGoAIML
Salary Range
$102,000 - $138,000 /year
This salary is estimated based on similar roles. The actual salary may vary.
About the Role
We are looking for a GPU Networking Engineer to join our Model Performance team at Baseten. This role focuses on building the software fabric that unifies thousands of GPUs into a cohesive operating system, making RDMA a first-class building block in our infrastructure.
Key Responsibilities
- →Integrate RDMA/RoCE/InfiniBand capabilities directly into our inference stack
- →Implement and tune networking layers for efficient Disaggregated KV Cache Offload and WideEP
- →Work with checkpointing and storage mechanisms to enable sub-10-second startup for trillion-parameter models
- →Characterize and validate networking performance on bleeding-edge clusters
- →Design tools for visualizing packet flow, congestion, and effective bandwidth
- →Optimize communication kernels to overlap compute and data transfer
Requirements
- ✓Deep experience with high-performance networking protocols (InfiniBand, RoCE v2)
- ✓Fluency in C++ or Python with ability to bridge high-level logic and hardware
- ✓Understanding of memory hierarchy in modern NVIDIA architectures
- ✓Experience diving into TensorRT-LLM source code and writing custom C++/Python bindings
- ✓Ability to determine when to use off-the-shelf solutions versus building custom solutions
Benefits
- ★Competitive compensation, including meaningful equity
- ★100% coverage of medical, dental, and vision insurance for employee and dependents
- ★Generous PTO policy including company-wide Winter Break
- ★Paid parental leave
- ★Company-facilitated 401(k)
- ★Exposure to a variety of ML startups
Free to sign up. Takes 30 seconds.
