Enhance the robustness of tests

[FarmersWrap]

json-io version

4.19.1

Java version

openjdk version "11.0.20.1"

Used tools

Apache Maven 3.6.3
Ubuntu 20.04.6 LTS
Linux version: 5.4.0-163-generic

Description

There are 9 tests within json-io has been reported as flaky when run with the NonDextool.

• Since hashmap in Java is unordered, we simulate the randomness of the hashmap by shuffling it to test the correctness of the tests in json-io. There may be more than one json string when hashmap is used.
  For example: com.cedarsoftware.util.io.AtomicLongTest,testAssignAtomicLong

• Tests should not rely on the getDeclaredFields() function because according to Class.getDeclaredFields() "The elements in the returned array are not sorted and are not in any particular order.". The test answers can differ.
  For example: com.cedarsoftware.util.io.MapsTest,testMap

Hash, class, test
f8958ef52bbdc94f8aed7d98c96d2775200ec7e7, com.cedarsoftware.util.io.AtomicLongTest, testAssignAtomicLong
f8958ef52bbdc94f8aed7d98c96d2775200ec7e7, com.cedarsoftware.util.io.AtomicBooleanTest, testAssignAtomicBoolean
f8958ef52bbdc94f8aed7d98c96d2775200ec7e7, com.cedarsoftware.util.io.MapsTest, testMap
f8958ef52bbdc94f8aed7d98c96d2775200ec7e7, com.cedarsoftware.util.io.MapsTest, testReconstituteMap
f8958ef52bbdc94f8aed7d98c96d2775200ec7e7, com.cedarsoftware.util.io.NoTypeTest, testNoType
f8958ef52bbdc94f8aed7d98c96d2775200ec7e7, com.cedarsoftware.util.io.PrettyPrintTest, testPrettyPrint
f8958ef52bbdc94f8aed7d98c96d2775200ec7e7, com.cedarsoftware.util.io.AtomicIntegerTest, testAssignAtomicInteger
f8958ef52bbdc94f8aed7d98c96d2775200ec7e7, com.cedarsoftware.util.io.EnumTests,testEnumWithPrivateMembersAsField_withPrivatesOn
f8958ef52bbdc94f8aed7d98c96d2775200ec7e7, com.cedarsoftware.util.io.CollectionTests, testEnumsInsideOfACollection_whenWritingAsObject_withPrivateMembersIncluded

Expected behavior

Test should pass even if the order in json string differs.

Actual behavior

The tests assume there will only one json string produced by TestUtil.toJson function. But after shuffling, they failed.

Reproduction steps

1. read instructions here NonDex
2. Execute following commands

#!/bin/bash
log_dir="jsonio-logs1"
# Create logs directory if it doesn't exist
mkdir -p ../$log_dir
# Perform Maven install, skip tests, and log the output in ../logs directory
mvn install -pl "." -am -DskipTests |& tee "../$log_dir/mvn-install-$(date +%s).nondex_log"
# Run Maven tests and log the output in ../logs directory
mvn -pl "." test |& tee "../$log_dir/mvn-test-$(date +%s).nondex_log"
# Execute NonDex Maven plugin
mvn -pl "." edu.illinois:nondex-maven-plugin:2.1.1:nondex -fn -DnondexRuns=10 |& tee "../$log_dir/mvn-nondex-$(date +%s).nondex_log"

Full description of Nondex please refer to its doc on github.

Thanks