Unbreak test suites with multiple files loading common constants

[debarshiray]

Test suites with a helper file with common constants that is loaded by
multiple files stopped working with:

  $ bats /some/path
  /some/path/test_helper.bash: line 1: A_CONSTANT: readonly variable
  Error while sourcing library loader at '/some/path/test_helper.bash'

If the helper file had anything else other than the common constants,
then the tests would run, but the spew about the constants would still
remain:

  $ bats --tap /some/path
  /some/path/test_helper.bash: line 1: A_CONSTANT: readonly variable
  1..2
  ok 1 first
  ok 2 second

Fallout from 9d5ecdb

• I have reviewed the Contributor Guidelines.
• I have reviewed the Code of Conduct and agree to abide by it

[debarshiray]

This goes on top of #902

[martin-schulze-vireso]

Thanks for your report and suggestion. I believe that we should not ignore load errors during the test registration, that will only lead to hard to diagnose problems.

IMHO, the correct way forward should be to separate the evaluationen of individual test files during test registration in the same way as is done for test execution.

[debarshiray]

Thanks for your report and suggestion. I believe that we should not ignore load errors during the test registration, that will only lead to hard to diagnose problems.

Yeah, I can imagine.

IMHO, the correct way forward should be to separate the evaluationen of individual test files during test registration in the same way as is done for test execution.

I don't understand the Bats code well enough to know how exactly that should be done. I can try it, if you give me some pointers, or I don't mind if you do it yourself and I hope that the tests would still be useful in that case. :)

[debarshiray]

Thanks for the review!

[debarshiray]

Tried to fix the changelog and shellcheck tests.

[martin-schulze-vireso]

I underestimated the complexity. The code for counting tests and duplicates relied on having everything evaluated in the same shell. This will need a rework but I am out of time for today.

[debarshiray]

I underestimated the complexity. The code for counting tests and duplicates relied on having everything evaluated in the same shell. This will need a rework but I am out of time for today.

Thanks for working on this, @martin-schulze-vireso !