New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decompressing the downloaded tar file is very slow #421

Open

Nastu-Ho opened this issue Apr 17, 2024 · 1 comment

Nastu-Ho commented Apr 17, 2024

Could you please provide a method to quickly decompress a large number of tar files?

Owner

rom1504 commented Apr 17, 2024 via email

The point of downloading as tar files is to use that directly during training with webdataset lib

On Wed, Apr 17, 2024, 2:45 PM Nastu Ho ***@***.***> wrote: Could you please provide a method to quickly decompress a large number of tar files? — Reply to this email directly, view it on GitHub <#421>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAR437UUVVTFAFCI73JYCCTY5ZVHDAVCNFSM6AAAAABGLH5EYKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGI2DQMRQHA3TENI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment