-
Notifications
You must be signed in to change notification settings - Fork 647
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
REFACTOR: remove redefinition of _row_lengths
and _column_widths
functions in PandasOnDaskDataframe
.
#3780
Comments
TBH, I would rather we add a specific Ray implementation that also gathers all objects widths in one go instead of doing a loop, this could potentially be a little bit faster. |
It should be significantly faster @vnlitvinov. |
…funcs in 'PandasOnDaskDataframe' Signed-off-by: Anatoly Myachev <[email protected]>
I wonder if #4683 and #4718 actually address this. @anmyachev thoughts? |
Really! Now the default implementation is enough, which for both Dask and Ray gather all remote objects at once. |
…3781) Signed-off-by: Anatoly Myachev <[email protected]>
Basic implementation should work for
Dask
engine. It already works forRay
engine.Links to basic implementations:
modin/modin/core/dataframe/pandas/dataframe/dataframe.py
Line 105 in d590de0
modin/modin/core/dataframe/pandas/dataframe/dataframe.py
Line 124 in d590de0
Links to
Dask
redefinitions:modin/modin/core/execution/dask/implementations/pandas_on_dask/dataframe/dataframe.py
Line 47 in d590de0
modin/modin/core/execution/dask/implementations/pandas_on_dask/dataframe/dataframe.py
Line 64 in d590de0
The text was updated successfully, but these errors were encountered: