Serialize using mapstructure conversion instead of JSON marshalling. #1401
Conversation
These changes remove the JSON encoding/decoding steps that are
performed during the serialization of ini and dotenv files. This
roundtrip loses type information during the transformation which
causes values to be incorrectly converted to the JSON marshaller
defaults (int becomes float64, bool becomes string, etc, etc). In
place of this JSON encoding, the mapstructure library allows for a
direct conversion between the Metadata struct and
map[string]interface{} needed to leverage the stores.Flatten and
stores.Unflatten functions.
In addition this adds mapstructure tags to the metadata structures to
allow backwards compatibility with the JSON encoding.
Resolves getsops#879 & resolves getsops#857
This is a re-submission of @acastle's getsops#1009 with mapstructure updated
to v1.5.0 and unused imports removed applied to getsops/sops/main HEAD.
| // to a map suitable for use with the Flatten function. | ||
| // | ||
| // Note: this will only emit the public fields of a structure, private fields are ignored entirely. | ||
| func ConvertStructToMap(input interface{}) (map[string]interface{}, error) { |
There was a problem hiding this comment.
This function is currently not used.
You need to:
- modify
FlattenMetadatain flatten.go to use this function; - modify
UnflattenMetadatain flatten.go to usemapstructure.WeakDecode; - remove the calls to
DecodeNonStrings, since they should be no longer needed.
Probably EncodeNonStrings should be changed as well to traverse the structure and call ValueToString for every non-string value, so that EncodeNonStrings no longer needs to know about the exact metadata structure.
There was a problem hiding this comment.
Oops! Restored ConvertStructToMap and WeakDecode per original patch.
Removed DecodeNonStrings, but TestDecodeNonStrings fails when:
err := DecodeNonStrings(tt.input)is replaced with just:
_, err := UnflattenMetadata(tt.input)So restoring DecodeNonStrings for now...
Presently, EncodeNonStrings considers only two cases: mac_only_encrypted and shamir_threshold. Are you thinking of replacing that with something like:
for k, v := range m {
if _, ok := v.(string); !ok {
m[k] = ValueToString(v)
}
}?
|
If following commits look okay, I'll squash and add sign-off per devstein request. |
slewsys
force-pushed
the
main
branch
6 times, most recently
from
d810639
to
de0505c
Compare
Restore WeakDecode to UnflattenMetadata.
Remove unused privateKeySizeLimit.
These changes remove the JSON encoding/decoding steps that are performed during the serialization of ini and dotenv files. This roundtrip loses type information during the transformation which causes values to be incorrectly converted to the JSON marshaller defaults (
intbecomesfloat64,boolbecomesstring, etc, etc). In place of this JSON encoding, the mapstructure library allows for a direct conversion between theMetadata structandmap[string]interface{}needed to leverage thestores.Flattenandstores.Unflattenfunctions.In addition, this adds mapstructure tags to the metadata structures to allow backwards compatibility with the JSON encoding.
Resolves #879 & resolves #857
This is a re-submission of @acastle's #1009 - with mapstructure updated to v1.5.0 and unused imports removed - applied to getsops/sops/main HEAD (previously #1338).