Lazy node attributes aren't "un-lazied" in some contexts

[ericnorris]

Description

After #10345, it is possible to use lazy in node attributes and they are "un-lazied" when accessed. Unfortunately there are some cases where cookbooks may access attributes in such a way that they are not evaluated automatically, and so we'll see things like #<Chef::DelayedEvaluator:0x000000010813cbb8 ... in rendered templates, files, etc., instead of the evaluated value.

Some examples that may trigger this behavior include: calling Enumerable methods like node['foo'].map, calling node['foo'].to_h, or calling JSON.pretty_generate(node['foo']). These methods may be invoked by third-party cookbooks, so we have little control over changing their behavior without forking the cookbook entirely, which is undesirable.

I'm filing this as a bug as I believe that the spirit of #10345 is to allow lazy attributes to work transparently, since the [] methods were implemented so that cookbooks that use [] to access node attributes did not need to be modified to "un-lazy" the value. That said, I realize that might not be straightforward to achieve - there are a lot of ways to access node attributes - so I'm happy to discuss it.

I've brainstormed some things that we could consider:

• Add to_h, to_json, map, etc. methods on ImmutableMash and ImmutableArray that evaluate any Chef::DelayedEvaluators before returning. This may be annoying as there are a lot of methods to cover.
• Walk the entire Node object and evaluate all Chef::DelayedEvaluators present after the compilation phase, but before convergence. Maybe this would be too slow?
• Add to_s, to_json, methods on Chef::DelayedEvaluator. This may have unintended consequences.

Chef Version

I believe this happens in all versions of Chef client version, but I'm able to reproduce this locally with:

$  chef-client --version
Chef Infra Client: 18.2.7

Platform Version

I've reproduced this on:

• Ubuntu 22.04
• MacOS Monterey

Replication Case

I'm including a couple of the examples that I mentioned above, but essentially anything you can think of that accesses the node attribute values without invoking [] will trigger the behavior:

In attributes/default.rb:

node.default['foo'] = {
  'bar' => 'baz',
  'qux' => lazy { "qux-#{node['foo']['bar']}" },
}

In recipes/default.rb:

def convert_to_flags(h)
  h.map { |k, v| "-#{k}=#{v}" }.join(' ')
end

# works
Chef::Log.info "node['foo']['qux']: #{node['foo']['qux']}"

# prints out `#<Chef::DelayedEvaluator ...`
Chef::Log.info "JSON.pretty_generate(node['foo']): #{JSON.pretty_generate(node['foo'])}"

# prints out `#<Chef::DelayedEvaluator ...`
Chef::Log.info "convert_to_flags(node['foo']: #{convert_to_flags(node['foo'])}"

Client Output

[2024-01-02T12:48:18-05:00] INFO: node['foo']['qux']: qux-baz
[2024-01-02T12:48:18-05:00] INFO: JSON.pretty_generate(node['foo']): {
  "bar": "baz",
  "qux": "#<Chef::DelayedEvaluator:0x0000000107e45658 .chef/local-mode-cache/cache/cookbooks/scratch/attributes/default.rb:3>"
}
[2024-01-02T12:48:18-05:00] INFO: convert_to_flags(node['foo']: -bar=baz -qux=#<Chef::DelayedEvaluator:0x0000000107e45658 .chef/local-mode-cache/cache/cookbooks/scratch/attributes/default.rb:3>

Stacktrace

N/A