Deprecate Skopt Suggestion Service

[tenzen-y]

/kind feature

Describe the solution you'd like
[A clear and concise description of what you want to happen.]
Unfortunately, the scikit-optimize was closed on February 29, 2024.

https://github.com/scikit-optimize/scikit-optimize

So we need to stop supporting the skopt suggestion service as well.
I think that we can follow similar deprecation steps as the MXNet.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

---

Love this feature? Give it a 👍 We prioritize the features with the most 👍

[tenzen-y]

@kubeflow/wg-automl-leads WDYT?

[PeterWrighten]

May I work for this issue?
/assign

[tenzen-y]

/unassign @PeterWrighten

We need to follow the deprecation flow step by step.

[PeterWrighten]

/unassign @PeterWrighten

We need to follow the deprecation flow step by step.

Got it. Thanks!

[andreyvelich]

That is sad to hear @tenzen-y.

Do we have any alternative to use Bayesian Optimization in Hyperopt or Optuna ?

[tenzen-y]

That is sad to hear @tenzen-y.

Do we have any alternative to use Bayesian Optimization in Hyperopt or Optuna ?

Unfortunately, both optuna and Hyperopt don't provide Bayesian sampler.

• https://optuna.readthedocs.io/en/stable/reference/samplers/index.html
• https://hyperopt.github.io/hyperopt/#algorithms

[tenzen-y]

If we want to provide Bayesian sample, we need to implement it ourselves.

[andreyvelich]

Maybe we could check some Open Source BO implementations.
For example: https://github.com/bayesian-optimization/BayesianOptimization

Also, quite recently @contramundum53 added GPSampler support in Optuna: optuna/optuna#5185

Maybe we can re-use it.
cc @c-bata @a9p for your ideas here.

[tenzen-y]

Also, quite recently @contramundum53 added GPSampler support in Optuna: optuna/optuna#5185

Oh, I didn't find that.

[tenzen-y]

So, setting optuna as a default Bayesian sampler might be better.

[contramundum53]

@andreyvelich @tenzen-y I'm glad that you are considering integration of Optuna's GPSampler into katib!
GPSampler is planned to be released in Optuna v3.6, our next minor release. It is still experimental and future breaking changes/bugs are possible (the code is only a month old!), but if it's okay for you, we'll be happy to hear that.

[tenzen-y]

@andreyvelich @tenzen-y I'm glad that you are considering integration of Optuna's GPSampler into katib! GPSampler is planned to be released in Optuna v3.6, our next minor release. It is still experimental and future breaking changes/bugs are possible (the code is only a month old!), but if it's okay for you, we'll be happy to hear that.

Oh, I see. Thank you for letting us know @contramundum53
Is there a graduation plan for GPSampler? For example, did the optuna community decide when GPSampler graduates to stable?

[contramundum53]

Sorry no, that will depend on many factors, such as how much the new feature is used, how many feature requests come, how many bugs we encounter, and so on.

[andreyvelich]

Thanks for letting us know @contramundum53!

@tenzen-y @johnugeorge What do you think about it ? Should we use skopt for 1 Katib/Kubeflow release and add deprecation notice for users who will run experiments with bayesianoptimization algorithm ?
Also, implement GPSampler in Optuna Katib Suggestion and explain users how to update Katib Config to use BO from Optuna rather than from skopt ?
After some time we will remove skopt and fully rely on Optuna for bayesianoptimization algorithm ?

[tenzen-y]

Sorry no, that will depend on many factors, such as how much the new feature is used, how many feature requests come, how many bugs we encounter, and so on.

@contramundum53 Thank you for letting me know!

[tenzen-y]

@tenzen-y @johnugeorge What do you think about it ? Should we use skopt for 1 Katib/Kubeflow release and add deprecation notice for users who will run experiments with bayesianoptimization algorithm ?
Also, implement GPSampler in Optuna Katib Suggestion and explain users how to update Katib Config to use BO from Optuna rather than from skopt ?
After some time we will remove skopt and fully rely on Optuna for bayesianoptimization algorithm ?

@andreyvelich I totally agree with you.

[tenzen-y]

/remove-release 0.16

[andreyvelich]

@tenzen-y Should we try to add deprecation notice before the next release ?

[tenzen-y]

@tenzen-y Should we try to add deprecation notice before the next release ?

@andreyvelich Shouldn't be marked as release 0.17, not 0.16?

[andreyvelich]

Oh, sorry, my bad

[tenzen-y]

Oh, sorry, my bad

no worries

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[tenzen-y]

/remove-lifecycle stale