You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Please consider adding PartitionSHAP
(or another fast attribution method)
Motivation
The currently available attribution methods run very long for large models like llama2 or mistral.
Pitch
The PartitionSHAP implementation from shap runs only minutes on a model like Mistral or GPT-2.
It is their best-performing explainer (runtime-wise), especially for text generation. It runs significantly quicker than any other method. The attributions are a bit less accurate the performance is very good.
As far as I know, it's the fastest model-agnostic explanation approach. Anything else using owen values should also be very fast.
Alternatives
Any other super quick attribution method would be very welcome 馃槃
I've listed fastSHAP below, which is also very fast but not proven on any LLMs.
Additional context
This is the PartitionSHAP implementation from the shap package. There's also fastSHAP, though I am not sure how applicable it would be to LLMs.
The text was updated successfully, but these errors were encountered:
LennardZuendorf
changed the title
Add PartitionSHAP (owen based calculation of SHAP values)
Add PartitionSHAP or other fast attribution method
Jan 2, 2024
馃殌 Feature
Please consider adding PartitionSHAP
(or another fast attribution method)
Motivation
The currently available attribution methods run very long for large models like llama2 or mistral.
Pitch
The PartitionSHAP implementation from shap runs only minutes on a model like Mistral or GPT-2.
It is their best-performing explainer (runtime-wise), especially for text generation. It runs significantly quicker than any other method. The attributions are a bit less accurate the performance is very good.
As far as I know, it's the fastest model-agnostic explanation approach. Anything else using owen values should also be very fast.
Alternatives
Additional context
This is the PartitionSHAP implementation from the shap package. There's also fastSHAP, though I am not sure how applicable it would be to LLMs.
The text was updated successfully, but these errors were encountered: