Is there a way to get the p-values when calling select_features?

[TheFilS]

As per title, I'm really interested in getting the p-values when select_features decides on top X features and rank orders them.
Alternatively, is there another way to get similar info from the decision based on the "power" of the feature as a classifier?

I couldn't find a way - my apologies if this is somewhere in the manual

[nils-braun]

Hi @TheFilS!
Maybe it would be beneficial to use the calculate_relevance_table function (https://tsfresh.readthedocs.io/en/stable/api/tsfresh.feature_selection.html#tsfresh.feature_selection.relevance.calculate_relevance_table). It is also used in the normal select features function and returns a dataframe with all the information, also including the p-value

[TheFilS]

Thanks I'll try it out! :)

[TheFilS]

@nils-braun Thanks again for this suggestion. I'm having troubling running it though.
For example, if I run the following, everything works ok:
features_filtered = select_features(combXidx, serclass)

it runs fine where:
combXidx is
value__variance_larger_than_standard_deviation
0 0.0
1 0.0
2 0.0
3 0.0
4 0.0
... ...
2839 0.0
2840 0.0
2841 0.0
2842 0.0
2843 0.0

  value__has_duplicate_max  value__has_duplicate_min  \

0 0.0 0.0
1 0.0 0.0
2 0.0 0.0
3 0.0 0.0
4 0.0 0.0
... ... ...
2839 0.0 0.0
2840 0.0 0.0
2841 0.0 0.0
2842 0.0 0.0
2843 1.0 1.0

...
[2844 rows x 784 columns]

And serclass is:
[1. 1. 1. ... 3. 3. 3.]
with a length of 2844

But, if I try to run the line for extracting the relevance table i get an error:
relevance_table = tsfresh.feature_selection.relevance.calculate_relevance_table(combXidx, serclass)

Error:

AttributeError Traceback (most recent call last)
Cell In[75], line 1
----> 1 relevance_table = tsfresh.feature_selection.relevance.calculate_relevance_table(combXidx, serclass)

File ~\envs\filsenv\lib\site-packages\tsfresh\feature_selection\relevance.py:163, in calculate_relevance_table(X, y, ml_task, multiclass, n_significant, n_jobs, show_warnings, chunksize, test_for_binary_target_binary_feature, test_for_binary_target_real_feature, test_for_real_target_binary_feature, test_for_real_target_real_feature, fdr_level, hypotheses_independent)
47 """
48 Calculate the relevance table for the features contained in feature matrix X with respect to target vector y.
49 The relevance table is calculated for the intended machine learning task ml_task.
(...)
159 :rtype: pandas.DataFrame
160 """
162 # Make sure X and y both have the exact same indices
--> 163 y = y.sort_index()
164 X = X.sort_index()
166 assert list(y.index) == list(X.index), "The index of X and y need to be the same"

AttributeError: 'numpy.ndarray' object has no attribute 'sort_index'

Any help would be greatly appreciated

[TheFilS]

To anyone who many run into the same problem as me, I had luck explicitly converting my array to a series (I didn't have to do this to call "extract_features" but had to do it to get "calculate_relevance_table" to work

Example:
serser = pd.Series(serclass)