Leaderworkerset v0.3.0

Features:

• RollingUpdate with MaxSurge support
• Subgroup support for disaggregated serving
• Example for multi-node serving of llama 70B on GPUs with vLLM
• Add a new start policy API
• Inject leader address environment variable to every container
• Spec.rolloutStrategy should be a non-required field

Acknowledgments

Thanks to our contributors in this release, in alphabetic order:
@ahg-g @Edwinhr716 @googs1025 @gujingit @jjk-g @kerthcet @liurupeng @nayihz

Leaderworkerset v0.2.0

Features:

• Support RollingUpdate with MaxUnavailable
• Allow Prometheus to gather metrics gathered by controller-runtime
• Fix TPU env var assignment when leader pod doesn't request TPU
• User guide to deploy multi-host inference with Saxml
• Increase qps limit for pod scheduling
• Setup E2E test and improve test coverage

Acknowledgments

Thanks to our contributors in this release, in alphabetic order:
@ahg-g @Bslabe123 @Edwinhr716 @googs1025 @kannon92 @kerthcet @liurupeng @nayihz @Zeel-Patel

Leaderworkerset v0.1.0

Features:

• Support creating groups of pods as a unit
• Support dual-template, one for leader and one for the workers
• Support autoscaling through HPA
• Support topology-aware placement
• Support all-or-nothing restart for failure handling

Acknowledgments

Thanks to our contributors in this release, in no particular order:
@liurupeng @Edwinhr716 @kerthcet @ahg-g