Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support basic xarray-functionality #1183

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

mrfh92
Copy link
Collaborator

@mrfh92 mrfh92 commented Jul 26, 2023

Description

In this PR we add a class DXarray that is somehow similar to xarray.DataArray but with heat.DNDarray's instead of numpy arrays; whenever possible, we use existing Heat-operations or process-local xarray-operations.

So far implemented:

  • class DXarray
  • interoperability: .xarray() (DXarray to xarray), from_xarray() (xarray to DXarray)
  • basic array manipulation: resplit_ (in place resplit of DXarray), balance_ (in place balacing of DXarray)
  • miscellaneous: print

To be done (for the beginning):

  • I/O (which formats?)
  • basic statistics, or --more general-- functions that operate on the data part of a DXarray only
  • checkout how more complex operations requiring interaction of data part and coordinates (e.g. interpolation etc.) can be done using xarray as process-local compute engine
  • how to handle date-time-coordinates (pandas data frames)?
  • ...

Due Diligence

  • General:
    • base branch must be main for new features, latest release branch (e.g. release/1.3.x) for bug fixes
    • title of the PR is suitable to appear in the Release Notes
  • Implementation:
    • unit tests: all split configurations tested
    • unit tests: multiple dtypes tested
    • documentation updated where needed

@mrfh92 mrfh92 linked an issue Jul 26, 2023 that may be closed by this pull request
@mrfh92 mrfh92 added this to the 1.4.0 milestone Jul 26, 2023
@ghost
Copy link

ghost commented Jul 26, 2023

👇 Click on the image for a new way to code review

Review these changes using an interactive CodeSee Map

Legend

CodeSee Map legend

@mrfh92
Copy link
Collaborator Author

mrfh92 commented Jul 26, 2023

@mtar I tried to change benchmark_main.yml, benchmark_pr.yml, and ci.yml in order to install xarray in the CI, but it did not work as expected.

@github-actions
Copy link
Contributor

Thank you for the PR!

@codecov
Copy link

codecov bot commented Jul 27, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.10%. Comparing base (1263887) to head (960bf6f).
Report is 482 commits behind head on main.

Current head 960bf6f differs from pull request most recent head 23e5cc6

Please upload reports for the commit 23e5cc6 to get more accurate results.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1183      +/-   ##
==========================================
- Coverage   91.83%   90.10%   -1.74%     
==========================================
  Files          77       79       +2     
  Lines       11191    10952     -239     
==========================================
- Hits        10277     9868     -409     
- Misses        914     1084     +170     
Flag Coverage Δ
unit 90.10% <0.00%> (-1.74%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mrfh92 mrfh92 mentioned this pull request Aug 21, 2023
@github-actions
Copy link
Contributor

Thank you for the PR!

1 similar comment
@github-actions
Copy link
Contributor

Thank you for the PR!

Copy link
Contributor

This pull request is stale because it has been open for 60 days with no activity.

Copy link
Contributor

This pull request is stale because it has been open for 60 days with no activity.

@github-actions github-actions bot added the stale label Jun 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support distribution of xarray
2 participants