When plotting the shap text it is showing an extra letter(Ġ) before every word.

[shafikrony]

Issue Description

From May 14th, it is showing an extra letter(Ġ) before every word in a sentence when using shap.plots.text(shap_values)

Code snippet:

pred = transformers.pipeline(
"text-classification",
model=model,
tokenizer=tokenizer,
device=0,
return_all_scores=True,
)
explainer = shap.Explainer(pred)
shap_values = explainer(df["text"][33:43])
shap.plots.text(shap_values)

For example main text:
I think this broke OAuth2, diaspora-client seems to dislike oauth2 0.5"
Showd text:
I Ġthink Ġthis Ġbroke ĠO Auth 2 , Ġdi as pora - client Ġseems Ġto Ġdislike Ġo auth 2 Ġ0 . 5 "

Minimal Reproducible Example

pred = transformers.pipeline(
    "text-classification",
    model=model,
    tokenizer=tokenizer,
    device=0,
    return_all_scores=True,
)
explainer = shap.Explainer(pred)
shap_values = explainer(df["text"][33:43])
shap.plots.text(shap_values)

Traceback

No response

Expected Behavior

Shouldn't show extra letter (Ġ)

Shouldn't

Bug report checklist

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest release of shap.
• I have confirmed this bug exists on the master branch of shap.
• I'd be interested in making a PR to fix this bug

Installed Versions

0.45.2.dev2

[CloseChoice]

Thanks for reporting. Your example is not reproducible, please provide a reproducible example I am afraid otherwise we wont have the capacity to figure out a model and a dataset that reproduces the issue.

Concretely we would need one script that reproduces your error, that means that the model definition, training steps, etc. and the data definition is all done within that script and does not have dependencies to any internal code/data of yours.