Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(polars): support to_torch #9159

Open
1 task done
ianmcook opened this issue May 8, 2024 · 3 comments
Open
1 task done

feat(polars): support to_torch #9159

ianmcook opened this issue May 8, 2024 · 3 comments
Labels
feature Features or general enhancements polars The polars backend

Comments

@ianmcook
Copy link
Contributor

ianmcook commented May 8, 2024

I see Polars recently added to_torch:
https://www.linkedin.com/posts/pola-rs_export-your-polars-dataframe-directly-to-activity-7193568311769542656-kqRz

I see Ibis has a to_torch API: #6518

Is the former supported via the latter?

Code of Conduct

  • I agree to follow this project's Code of Conduct
@ianmcook ianmcook added the feature Features or general enhancements label May 8, 2024
@gforsyth
Copy link
Member

gforsyth commented May 8, 2024

It is not, currently.

We go from any backend (including polars) -> pyarrow -> numpy -> torch.from_numpy

Polars has it's own fast (er?) path from a polars frame to a numpy array, then also uses torch.from_numpy.

@ianmcook
Copy link
Contributor Author

ianmcook commented May 8, 2024

Got it, thanks. Yeah, I don't know if it's worthwhile to directly map Ibis to_torch to Polars to_torch for performance or other reasons.

@gforsyth gforsyth added polars The polars backend onboarding Issues that can be addressed by someone less familiar with ibis and removed onboarding Issues that can be addressed by someone less familiar with ibis labels May 10, 2024
@alexander-beedie
Copy link
Contributor

Author of the Polars to_torch export here... For the curious, the advantage in a native export path wouldn't be performance (both paths require only a single copy per-array, with no conversion overheads), it would be in avoiding the pyarrow dependency. However, if you've already eaten that import cost then there's no advantage to be had by adding a native path (aside from access to export modes other than "dict").

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Features or general enhancements polars The polars backend
Projects
Status: backlog
Development

No branches or pull requests

3 participants