magic_duckdb based magic?

[eitsupi]

In combination with DuckDB, it seems that magic_duckdb may provide better performance than jupysql.
https://github.com/iqmo-org/magic_duckdb

[max-sixty]

Yes, looks interesting. I'm not sure whether it's sufficiently mature or not — I notice it's new and has unfinished sentences on the Readme :) (all tools start somewhere!).

The perf does look much better though, I'm surprised the existing approach has so much overhead

[eitsupi]

jupysql is really amazing. This performance issue should be solved...

ploomber/jupysql#470
https://twitter.com/ploomber/status/1661824499224915970
https://github.com/ploomber/jupysql/blob/master/CHANGELOG.md

0.7.5 (2023-05-24)
[Feature] Using native DuckDB .df() method when using autopandas

[edublancas]

yes, we fixed the performance issue when %autopandas is turned on. And we just started working on ploomber/jupysql#470 to provide further improvements.

If you encounter any performance issues, let us know and we'll fix them!

[max-sixty]

Thank you @edublancas !