-
Notifications
You must be signed in to change notification settings - Fork 145
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bookmark potential integration target for ZFP #193
Comments
@markcmiller86 This looks interesting but I'm not sure how to best expose zfp through such an API since zfp fundamentally operates on multidimensional arrays, whereas the API seems geared toward streams of numbers. The I/O stream would somehow have to know about array dimensions and/or strides for this to work well. There's a similar problem of specifying compression settings, e.g., error tolerance. I've not had a chance to take a close enough look at the I/O library to see if there's an easy way to incorporate that. Perhaps one could use something similar to the C++ stream manipulators like The only other alternative I can think of is to have the user reorganize the data into contiguous blocks, but even then zfp wouldn't know what the dimensionality of the data is (1D, 2D, 3D, ...). |
I happened to run across this C/C++ stream io utility suite which apparently already integrates zlib, gzip and zstd and I am thinking it could be a potential place to integrate ZFP as well. I suspect they are doing only byte-level compression in their implementations and not doing numeric and/or array-level compression but I've always wondered what it would take (either in the way of stream manipulators or whatever) to extend C++ iostream interface to support ZFP compression.
So, this isn't really an issue...just a bookmark for possible future work.
The text was updated successfully, but these errors were encountered: