Null values are omitted when publishing events

[diastremskii]

By default, Kanadi omits all null values from the event JSON when publishing events because of the default configuration.

For example, when using a schema like

{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "title": "Event",
    "type": "object",
    "properties": {
        "test": {
            "type": [
                "string",
                "null"
            ]
        }
    },
    "required": [
        "test"
    ]
}

An event with data {"test": null} is going to be successfully published when calling Nakadi directly, but it's going to fail when publishing through Kanadi, because the field test is going to be dropped.

[gchudnov]

Hi @TheBeastOfCaerbannog
thanks for reporting an issue.

true, it might worth to add a way to configure null-propagation (?)

If we just enable null-propagation (without configuration), it might break the current behavior some apps might be relying on now, where null-values are not propagated.

Another option could be - split the library and extract encoding / decoding but that will require lots of changes.
So will check the first option -- null-configuration.

If you have any other ideas, please let me know.

Thank you,
Grigorii

[diastremskii]

Hi @gchudnov,
Thank you for checking that on such short notice!

Yeah, definitely, giving an option to configure that would be great and would be a backward-compatible change if we keep the current configuration by default.

I think this logic was added to omit null's from metadata/some other parts of the event (except the event data), because Nakadi doesn't accept null's for some of the fields in metadata. But because Parser is recursive, the unexpected effect was that it also omits null's from the event data itself.