New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
YAML anchors, aliases and references #285
Comments
In my work on ordered maps I am writing custom logic around yaml encoding/decoding. I expect I'll be able to resolve this issue when reading but writing could result in strange results - probably removal of the alias/merge and duplicate data The reason behind this is that dasel, outside of encoding/decoding is unaware of these concepts. I think I will purposely disallow this for now until I can come up with a proper solution to keep context throughout the lifecycle. |
@TomWright What do you think of solving the issue partially? i.e. implementing only the reading? |
Perhaps behind a flag. I don't want people to accidentally remove all aliases from their files |
I wish dasel would preserve in the output the aliases, anchors, references, order and comments from the input $ echo '
foo: foo
baz: baz
bar: &bar
dasel: rules
cheese:
<<: *bar
# comment
' | dasel -r yaml
bar:
dasel: rules
baz: baz
cheese:
dasel: rules
foo: foo With that being said, I don't think dasel should care, since dasel is mostly used to transform/manipulate the input through pipes (same as in the snippet above which is inspired by the docs) — piping in and piping out. |
The trouble comes because dasel decodes all input data into a generic format, before processing the data and encoding back into the desired format. This is how the cross-format transformations are possible. The issue is that the generic format doesn't know about aliases, references etc right now. The ideal solution is to obviously make it aware and deal with it properly when re-encoding but that isn't an easy task. That is what I plan to do, but it will take time. A requirement for this is that all data goes through some more in-depth decoding so I can actually read that information. The plus here is that the same is required for ordered maps and so some of the work will be done. |
So from what I see, there is no need to place it behind a flag as it preserve the current behavior of dasel.
By the way, I am unfamiliar with any tool that does this. |
@TomWright docker-compose is writing in go lang and has a support for anchors, aliases and references. It also does not suffer from things like #278, #294, #245, #293, etc. Maybe it worth taking a look at docker-compose and learn from it. |
I am in the middle of reworking the yaml processing dasel does. A good bunch of those issues are solved with the rework, and some of them just aren't yaml processing issues. All of the number formatting issues are due to the default output formatting of number values in golang, but I am thinking of solutions for those. The fact that dasel parses into generic types massively complicates things in comparison to something like docker-compose that will be unmarshaling into known structs with expected types. |
@TomWright I switched to a tool of my own which consumes structured files (json, yaml, csv, etc.) and let you manipulate them easily. I did it by reading the structured files, converting them to json and then apply a given manipulation (selection/transformation). I did not have to come up with any specific regular language for the manipulation, as I used the builtin syntax and command of the programming language that I used. I can elaborate more on that, but maybe dasel can benefit from the same approach. I have a feeling it will simplify things. |
@electriquo If you have a look to the code I'd be more than happy to take a look |
@TomWright the code is not polished to share it with the public. but let me share some short snippet $ cat grabber.rb
#!/usr/bin/env ruby
require 'yaml'
file = ARGV[0]
query = ARGV[1]
data = File.read(file)
parsed = YAML.safe_load(data, aliases: true)
result = eval("#{parsed}#{query}")
puts(parsed.inspect)
$ grabber.rb test.yaml '.keys().map {|e| e.upcase}' |
Is your feature request related to a problem? Please describe.
YAML supports anchors, aliases and references, but Dasel is unable to read it.
Describe the solution you'd like
Dasel should be able to render the YAML correctly
The text was updated successfully, but these errors were encountered: