Overwriting subconfig retains values from default subconfig

[cristianregep]

🐛 Bug report

I have a master config that is formed from multiple other sub-configs stored in separate files (one for data, one for architecture etc.). When I specify from the CLI a different version of a sub-config my expectation is that the sub-config gets completely overwritten. Instead, what I got is a merged sub-config between the default and the one provided through the CLI. I believe this is a bug. Otherwise let me know if this intended, and if maybe there is a setting through which we can control the behaviour to overwrite

To reproduce

test.py

from jsonargparse import CLI
from dataclasses import dataclass


@dataclass
class Data:
    attribute_1: int
    attribute_2: int


@dataclass
class Test:
    data: Data


print(CLI(Test, fail_untyped=True))

main_config.yaml

data: data_default.yaml

data_default.yaml

attribute_1: 1
attribute_2: 10

data_new.yaml

attribute_1: 2

python test.py --config main_config.yaml --data data_new.yaml

Expected behavior

I should get an error that says attribute_2 is required but has not been provided.

Instead, i get
Test(data=Data(attribute_1=2, attribute_2=10))

Environment

• jsonargparse version: 4.23.1
• Python version : 3.10.12
• How jsonargparse was installed: pip install jsonargparse[signatures,typing-extensions]==4.23.1
• OS (e.g., Linux): Ubuntu 22.04.2 LTS

[mauvilsa]

Config merging is the expected behavior and not a bug. Config files are one of the input types applied in the override-order. Only the settings that are present in a config are updated. It would be very confusing for users if what is not in the config gets changed.

Currently there is no way to discard values not given in the config. Also, I am not sure it is a good idea. If a value is discarded, then what value would it have? None? Or the default in the source code? Or after the overrides from default_config_files? There are several possibilities and it wouldn't be obvious to the user what to expect.

My advice is, be aware of the override order and that only what is given gets modified. If you want to modify all the values, then provide all the values. Better to be explicit than needing to remember some implicit behavior.