Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Modin should work correctly with pandas, which uses pyarrow as a backend #7203

Closed
anmyachev opened this issue Apr 19, 2024 · 1 comment · Fixed by #7204
Closed

Modin should work correctly with pandas, which uses pyarrow as a backend #7203

anmyachev opened this issue Apr 19, 2024 · 1 comment · Fixed by #7204
Assignees
@anmyachev
Labels
new feature/request 💬 Requests and pull requests for new features P1 Important tasks that we should complete soon

Comments

@anmyachev
Copy link
Collaborator

The main known issue is that Modin's type predetermination system is based on NumPy types.

Found in #7199.
@anmyachev anmyachev added new feature/request 💬 Requests and pull requests for new features P1 Important tasks that we should complete soon labels Apr 19, 2024
@anmyachev anmyachev self-assigned this Apr 19, 2024
@anmyachev
Copy link
Collaborator Author

The current state of affairs is that we do not use backend information anywhere. The main place where this is needed is when accessing the type, through fields Series.dtype and DataFrame.dtypes. The simplest way would be to obtain information from partitions, but this would lead to poorer performance.

anmyachev added a commit to anmyachev/modin that referenced this issue Apr 19, 2024
@anmyachev
FEAT-modin-project#7203: Make sure modin works correctly with pandas,…
b710865 
… which uses pyarrow as a backend

Signed-off-by: Anatoly Myachev <[email protected]>
This was referenced Apr 19, 2024
Merged
Merged
anmyachev added a commit that referenced this issue May 14, 2024
@anmyachev @YarShev
FEAT-#7203: Make sure Modin works correctly with pandas, which uses p…
aa6e5c2 
…yarrow as a backend (#7204)

Signed-off-by: Anatoly Myachev <[email protected]>
Co-authored-by: Iaroslav Igoshev <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@anmyachev anmyachev

Labels
new feature/request 💬 Requests and pull requests for new features P1 Important tasks that we should complete soon
Projects
None yet
Milestone
No milestone
1 participant
@anmyachev