Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Question] cross validation strategy/individual cross validation results? #1727

Open
quietscientist opened this issue Apr 21, 2024 · 0 comments

Comments

@quietscientist
Copy link

Is there a way to see 1) what cross validation strategy was selected and 2) what the individual cross validation results for a given model?

I am training a classifier model as follows:

automl = AutoSklearn2Classifier(
ensemble_size=1,
dataset_compression=False,
allow_string_features=False,
time_left_for_this_task=1200,
per_run_time_limit=30,
metric=autosklearn.metrics.balanced_accuracy,
delete_tmp_folder_after_terminate=False,
memory_limit=None,
disable_evaluator_output=False,
)

After training, I fit the model with automl.fit(X_train, y_train) and refitting with automl.refit(X_train.copy(),y_train.copy()).

I can see the "mean test score" in automl.cv_results, but can't find any info related to individual cv results that were used to compute that mean. I also tried using AutoSklearn 1, specifying the cv strategy and the number of folds, but can still only find the mean in the cv_result.

Any advice on where to look/how to get that information?

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant