How can we handle Semantic API related Negative test cases?

[sanjana]

When resolving Issue #28583, a question was raised on how can we write Semantic API-related negative test cases.
Such cases should always have an output and not an exception. Thus, we shall decide on an appropriate approach to test these negative cases.

[pubudu91]

What do you mean by "negative tests"? In the linked issue, that's a bug, not an expected behaviour.

[sanjana]

@pubudu91 wouldn't it better if such unexpected behaviors are captured and handled in the Language Server? In that case, we need to provide the user with an appropriate output rather than an exception. Please consider the test cases that consist of such outputs as the negative test cases in the above section.

@nadeeshaan Anything to add?

[nadeeshaan]

I believe there should be test cases, for such scenarios, but that should be coverage tests. If we are to handle all the possible negative cases, that will not scale in the long run.

[pubudu91]

I'm still a bit unclear on what you meant by "negative tests"/"unexpected behaviour". In the linked issue, is getting an NPE for vals. what you meant by unexpected behaviour?