Too many non-reproducible issues

[CloseChoice]

We've always had the problem (but quite recently this really stuck out to me) that a lot of reported issues do not have reproducible examples. It is quite hard from just an error message and to know which function was called to reverse engineer the error. Therefore I suggest two things:

1. make it clearer in the bug issue template that issues without a script that reproduces them have very low priority. Also let's make it clear that we need one script without external dependencies (in terms of model and data) that reproduces the issue.
2. add a label non-reproducible script (or maybe we'll find a better name) and label them consequently.

I tried to reproduce a couple of issues but it just costs so much time and I failed quite a number of times. Since we are short on contributor hours we should really focus to invest our time in issues that are well prepared and not waste our time in figuring out what the problem is another time, but load this burden onto the reporter.

Edit: just a couple of recent examples, none older than 1 month:

• When plotting the shap text it is showing an extra letter(Ġ) before every word.  #3660
• BUG: Additivity check failed #3657
• BUG: TypeError: ufunc 'isfinite' not supported for the input types #3656
• BUG: 'tuple' object has no attribute 'as_list' #3645
• BUG: Warning: unrecognized nn.Module: Chomp1d #3618
• BUG: TypeError: waterfall() got an unexpected keyword argument 'features' #3628

[CloseChoice]

@connortann your thoughts on this are more than welcome

[connortann]

I agree with your analysis, you're absolutely right. I think those two steps sound prudent.

I made a start tweaking the bug report template to draw attention to the importance of the MRE: #3686

[CloseChoice]

Thanks, I close this for now but will have an eye on the issue. Might open this up again if the problem persists, though we'll have to live with a certain number of non-reproducible issues