Skip to content

Commit

Permalink
Update kserve document. (#993)
Browse files Browse the repository at this point in the history
Signed-off-by: Syulin7 <[email protected]>
  • Loading branch information
Syulin7 committed Sep 18, 2023
1 parent a7e6a0f commit 4dc39d6
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions docs/serving/kserve/custom.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ deploy an InferenceService with a predictor that will load a bloom model with te
3\. Check the status of KServe job

$ arena serve list
NAME TYPE VERSION DESIRED AVAILABLE ADDRESS PORTS
bloom-560m KServe 00001 1 1 http://bloom-560m.default-group.example.com :80 1
NAME TYPE VERSION DESIRED AVAILABLE ADDRESS PORTS
bloom-560m KServe 00001 1 1 http://bloom-560m.default-group.example.com :80

$ arena serve get sklearn-iris
$ arena serve get bloom-560m
Name: bloom-560m
Namespace: default
Type: KServe
Expand Down Expand Up @@ -70,7 +70,7 @@ Add the canaryTrafficPercent field to the predictor component and update command
$ arena serve update kserve \
--name bloom-560m \
--canary-traffic-percent=10 \
"text-generation-launcher --disable-custom-kernels --model-id /mnt/models/bloom-560m-v2 --num-shard 1 -p 8036"
"text-generation-launcher --disable-custom-kernels --model-id /mnt/models/bloom-560m-v2 --num-shard 1 -p 8080"

After rolling out the canary model, traffic is split between the latest ready revision 2 and the previously rolled out revision 1.

Expand Down

0 comments on commit 4dc39d6

Please sign in to comment.