You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Since it was slow, I used python multiprocessing to calculate the shape values in chunks. But the shape values of each chunk is an object with base_values (which i assume is shap values), values, and data.
I was wondering how I can concatenate them to make one single shap object. here is my code:
explainer = shap.Explainer(predict_func, X_train, feature_names=feature_names)
def compute_shap(sample):
return explainer(sample)
# Split X_test_transformed into chunks to distribute across processes
chunk_size = len(X_test) // multiprocessing.cpu_count()
chunks = [X_test_transformed[i:i + chunk_size] for i in range(0, len(X_test_transformed), chunk_size)]
with multiprocessing.Pool(processes=multiprocessing.cpu_count()) as pool:
shap_values = pool.map(compute_shap, chunks)
Update
Is this a correct solution?
data = shap_values[0].data
base_values = shap_values[0].base_values
values = shap_values[0].values
shap_features = shap_values[0].feature_names
for i in range(1, len(shap_values)):
values = np.concatenate((values, shap_values[i].values), axis=0)
base_values = np.concatenate((base_values, shap_values[i].base_values), axis=0)
data = np.concatenate((data, shap_values[i].data), axis=0)
assert shap_features == shap_values[i].feature_names
shap.Explanation(values, base_values=base_values,data=data,feature_names=shap_features)
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hello,
Since it was slow, I used python multiprocessing to calculate the shape values in chunks. But the shape values of each chunk is an object with base_values (which i assume is shap values), values, and data.
I was wondering how I can concatenate them to make one single shap object. here is my code:
Update
Is this a correct solution?
best
Beta Was this translation helpful? Give feedback.
All reactions