-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
excessive memory usage in more #6397
Comments
note that we have been focusing on compatibility, not perf or memory usage for now (even if some programs are already doing better than GNU's) |
True, but I also agree that 15x the filesize is a bit excessive 😄 |
Even 100% would be excessive, consumed memory shouldn't scale linearly with the file size at all. |
It's also just extremely slow to load such a file. I'm looking into making this at least a bit better. |
For coreutils sensible memory usage is part of correctness, especially so for a tool like more. For example say I logged in to a production machine with the intent of running The gnu coreutils variant uses few MB of ram making itself a perfectly viable tool in that scenario. That is to say this code needs to deal with the file in chunks and at the moment is not fit for use whatsoever. Any work done on it which would be thrown away while making the adjustment is a waste of time, which would be better spent either reworking this or beating other tools to shape in a lasting manner. |
Not useful for you does not mean useless for everyone. We've already agreed this is an issue and want to fix it. Your point has been made. Edit: For a bit more context: there are many more parts to get right here that are independent of this issue: the keyboard input, the rendering, the argument parsing, etc. Reworking the way the buffer works can still be while other changes are being made. |
Running it over a test file of about 1G in size results in 15G RSS.
Problems start with slurping the entire thing upfront, I don't know how it manages to amass 15 x memory overhead on top of it.
more from gnu coreutils sits at around 2MB.
How to repro:
perl -e 'print "meh\n" x (256 * 1024 * 1024)' > testfile
cargo run -r testfile
The text was updated successfully, but these errors were encountered: