Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option to load UNSMRY data as 32 bit floats #797

Open
asnyv opened this issue Jan 18, 2021 · 1 comment
Open

Option to load UNSMRY data as 32 bit floats #797

asnyv opened this issue Jan 18, 2021 · 1 comment
Assignees

Comments

@asnyv
Copy link
Contributor

asnyv commented Jan 18, 2021

According to the Eclipse documentation, the vector parameter values in UNSMRY are stored with the data type REAL, which is described as single precision (4 byte) floating point reals.

In ecl, at least from the python side (think also C++ side, at least what I've seen), all values are returned as float64, giving a substantial memory overhead. For a single simulation, it doesn't have much to say, but if you aggregate to ensembles, the difference is significant.

Would be nice to be able to get float32 directly (as an optional flag to avoid when requesting e.g. numpy arrays or pandas DataFrames (preferably also on the C++ side to limit memory usage also on that side, though the python output of the bindings are the most important for my personal use).

@pinkwah
Copy link
Collaborator

pinkwah commented Feb 2, 2021

We did some Sherlock Holmes-ing. It seems very feasible. While the following isn't the only places where we end up while reading UNSMRY data, it seems to be one of the more important places. Notice that we're reading a float and casting it to double.

https://github.com/equinor/ecl/blob/master/lib/ecl/ecl_unsmry_loader.cpp#L76-L83

What remains is to figure out the other places (seems like a few caching and/or default values that also should be floats?) and also what to do with the current API. My hypothesis is that casting float -> double -> float is a lossless operation, in which case this can just be the default and we could provide the user with a different function, ie. pandas_frame32() or something for backwards compatibility, and then make 32-bit the default in ecl3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants