bugs polars: using NuDataFrameCustomValue inside with-column

[maxim-uvarov]

Describe the bug

I compiled the polars plugin with #12819.

And I found something new: previously, it was possible to use the $df.a notation inside dfr with-column. Now it is broken.
@ayax79, please take a look to see if it is possible to fix it.

How to reproduce

> let $df = [[a];[1]] | polars into-df

> let $df2 = [[a b]; [2 3]] | polars into-df

> $df | polars with-column ($df2.b)
Error: nu::shell::cant_convert

  × Can't convert to NuExpression.
   ╭─[entry #3:1:26]
 1 │ $df | polars with-column ($df2.b)
   ·                          ────┬───
   ·                              ╰── can't convert NuDataFrameCustomValue to NuExpression
   ╰────

Expected behavior

I expect polars to accept $df.a notation

Screenshots

No response

Configuration

key	value
version	0.93.1
major	0
minor	93
patch	1
branch	main
commit_hash	c54d223
build_os	macos-aarch64
build_target	aarch64-apple-darwin
rust_version	rustc 1.77.2 (25ef9e3d8 2024-04-09)
rust_channel	1.77.2-aarch64-apple-darwin
cargo_version	cargo 1.77.2 (e52e36006 2024-03-26)
build_time	2024-05-07 05:50:32 +00:00
build_rust_channel	release
allocator	mimalloc
features	dataframe, default, sqlite, system-clipboard, trash, which
installed_plugins	explore, image, inc, polars

Additional context

No response

[ayax79]

This was a behavior provided by eager dataframes. I am trying to determine if cross dataframe references can be handled with with_column in lazy dataframes.

[ayax79]

The example above can be achieved with:

let $df = [[a];[1]] | polars into-df
let $df2 = [[a b]; [2 3]] | polars into-df
$df | polars append $df2.b

Write now the dot syntax creates an eager dataframe and selects the column. I have considered repurposing the dot syntax for a synonym for polars col b in creating a column expression, but decided against it as it would break all cross dataframe operations like $df.a / $df.b etc.

With the lazy refactor all with-column expressions only work with one dataframe as that seems to be way that the polars lazy api itself works.

[maxim-uvarov]

I got it, thank you for the explanations. I will update (submit pr) the dataframes chapter in the book accordingly.

I believe it is also worth mentioning this in the 0.94 update notes about the polars transition to lazyframes.