Re-add schemaOnly parameter when applying raft log entries and ensure we reloadDB on catching up

[reyreaud-l]

What's being changed:

Reverts #4882. Re-add the schema only check in the db.apply function and add a new atomic dbAndRaftAligned that is set to true once RAFT has caught up and re-applied log entries after a restart and we no longer need to use schema only.

This is necessary because on restart RAFT will re-apply log entries from disk to update the in-memory representation. As our DB operations are not truly idempotent we want to use a schemaOnly parameter to avoid updating the DB and only update the in-memory schema representation.

For example given the following order of operations (Operation -> In memory SchemaState | DBState)

1. AddClass(c1)        -> classes=[c1] | object["c1"] = []
2. DeleteClass(c1)     -> classes=[]   | object[]     = []
3. AddClass(c1)        -> classes=[c1] | object["c1"] = []
4. AddObject(c1, obj1) -> classes=[c1] | object["c1"] = [obj1]

If a node is to restart, it will replay the raft log entries (which are 1, 2 and 3 above) but not the data related operations as these are not handled by RAFT. On restarting a node the following sequence is happening

0. Initial state            -> classes=[]   | object["c1"] = [obj1]
2. (Replay) AddClass(c1)    -> classes=[c1] | object["c1"] = [obj1]
3. (Replay) DeleteClass(c1) -> classes=[]   | object[]     = [] <-- Here we lose data by *also* dropping obj1
4. (Replay) AddClass(c1)    -> classes=[c1] | object[]     = []

With this change when we are replaying log operation we avoid updating the DB until we have caught up and then reload the DB representation to ensure we have everything loading as is represented in the up-to-date schema.

Review checklist

• Documentation has been updated, if necessary. Link to changed documentation:
• Chaos pipeline run or not necessary. Link to pipeline:
• All new code is covered by tests where it is reasonable.
• Performance tests have been run or not necessary.

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud