vimcat is much slower than vanilla nvim

[GordianDziwis]

Open a file in vim takes less than a second, using vimcat in a terminal takes about 9s to "scroll" to the end of the file. vimcat -o - .vimrc >> test takes about 7s.

This is a problem, because I would like to use vimcat as a previewer for ranger.
Ranger calls vimcat like this:

# wraps highlight to treat exit code 141 (killed by SIGPIPE) as success
safepipe() { "$@"; test $? = 0 -o $? = 141; }
try safepipe vimcat -u ~/.dotfiles/.vimpagerrc -o - "$path" && { dump | trim; exit 5; }

Does anyone has an idea?

[rkitover]

I will try to speed it up.

[nkh]

@rkitover any progress?

Also an explanation of why you think it's slow would be very helpful.

[rkitover]

I'm sorry I haven't been doing much work on this repo lately.

I need to speed up the streaming system, but it is essential for viewing large files or it would hang at the beginning.

I can do two things here easily:

• Check file size, this would not work with pipeline input. Well, perhaps it could if it buffers a chunk at the beginning and it's smaller than that chunk.
• Add a --no-stream option to disable streaming.

[nkh]

I played a bit with this today:

script -o 100000 -qfc "vim -R --cmd 'set shortmess+=sSF t_ti= t_te=' +redraw +q <file_name_here>"

Blasing fast but there's a few things to fix:

• displays a single page
• running it in fzf sometimes gave me short pages (24 lines)
• the short message that's at the bottom of the display is displayed at the top, nothing perl or sed can't fix
• some spurious message like "Top" added

I also tried grabbing the buffer via tmux, didn't go so well.

[rkitover]

The main problem with this approach is like you said, it's only the first page. vimcat can also output a single page very fast, especially if I set the first chunk to something bigger or turn off streaming.

Your approach may also work if you add some vimscript to page downwards through the whole file.

[rkitover]

Actually, now that I think about it, making the first chunk say, 10k, would mostly fix this problem, what do you think?

[rkitover]

I tried setting the chunk size to 1MiB, but it does not make it any faster.

What do you find annoying about it? It seems pretty fast to me, other than the fraction of a second pause at the very beginning.

[nkh]

I need to show a lot of file previews, lots. vimcat takes eons, 18 seconds to get 287 lines of bash, that a test I just ran. It could be that I have something wrong in my installation but then it was wrong on my last 3 computers. cCould you try and let me know what you get for time? https://raw.githubusercontent.com/nkh/ftl/main/ftl

I'd be delighted to use vimcat if it was faster, I used to alias it to cat but I've become impatient with age ;) I certainly hope it will be fast in the future so I can cat with the same colors as my editor, that's much more ergonomic.

As for the short code I posted above, It doesn't even list a whole page in an fzf preview, the spurious elements are a nuisance, speed is just fine. I'll have to spend more time to make it work properly, hopefully you beat me to it and vimcat becomes a speed cat.

[rkitover]

For your ftl script, I get:

real    0m6,996s
user    0m7,285s
sys     0m0,378s

. If I run vimcat on vimpager, which is a 1045 line script, I get:

real    0m2,538s
user    0m3,147s
sys     0m0,391s

. I suspect that the difference here is because your ftl script is extremely syntax-heavy, and either vim highlighting in general or the ANSI encoding I do becomes the bottleneck due to the number of syntax elements. Still, I get 7 seconds not 18.

If it would be possible to get your method to work, that would cut-out the ANSI re-encoding at least, which is done with vimscript, which is not exactly blazing fast.

[nkh]

https://www.reddit.com/r/vim/comments/vzs658/vimkat_a_rewrite_to_speedup_vimcat/

@rkitover, here's what I have, not polished yet but on its way there. I'm a bit surprised by the time difference between your vimcat run and mine (7s vs 18s), could it be that we use different versions of vimpager/vim? I get 0.5s with vimkat.

@BonaBeavis, maybe this fixes your problem too

[rkitover]

@nkh Awesome, I'll take a look.