Intersted in picture or video rendering implementation?

[meteor-lsw]

I have tried to render an image and the results look good.

I have submitted the relevant example code #141 .

Is there anyone else interested in rendering images or videos on the terminal? I think rendering at 240p resolution is feasible. If anyone else is interested in this topic, welcome to discuss and work towards this goal.

[meteor-lsw]

[fdehau]

Regarding, your two last examples I think those are high-res outputs are achieved using custom apis provided by some terminal emulators. Unless, one the backends in use within this crate supports it, it is unlikely that we will be able to reproduce them.

[cjbassi]

It seems like sixel graphics are gaining in popularity as a way to display images in terminals, especially with the help of libsixel which even has a couple of rust api bindings. Sixel is already supported by a few terminals including xterm (with a compile time flag and some configuration), and there's an open issue for it in alacritty here. Someone has also been working on adding sixel support to termui here.

[srlehn]

I think the first of longwusha's two example images is the ranger file manager displaying with w3mimgdisplay.
The second image seems to be sixel.

[samsieber]

I'm actually very interested in being able to display arbitrary image data, mostly from the image crate. That would make this crate very useful to me. I think I'm going to try using termion for now.

[EvanCarroll]

opened an issue in termion

https://gitlab.redox-os.org/redox-os/termion/issues/180

[ghost]

@samsieber

If you decide to implement sixel, then I wanted to pass on some things I've picked up recently, on the odd chance it could help. (Even if just by way of "go check out notcurses, it's really cool." ;) )

I implemented sixel output in my Java-language library in late 2018. For a while only xterm, RLogin, and yaft could handle the output without crashing. I reported on that work in a terminal emulators forum, and several folks reported bugs in a few terminals. Now there are at least 10-20 terminals and terminal widgets that work well.

Last week I added translucent windows with images under/over, thanks in large part to some clues from notcurses. Some notes of that work are here: https://gitlab.com/klamonte/jexer/-/issues/88 . I also wrote down more details on my particular mixed image-and-text cell model here , with some narrative (and again the notcurses shoutouts) here.

For encoding to sixel, check out notcurses' new octree implementation sometime. It's very likely the fastest and highest-quality anywhere right now, at least for xterm-type terminals. (libsixel was tested on real hardware, so that might give it an edge still in terms of compatibility.)

Depending on how you choose to render, you may find that 256-bit colors per sixel could still result in close to 16-bit apparent visual bit depth on the actual screen, making the drive for non-sixel support a bit less crucial. (Note that the best bit depth for sixel is 19.97 bits: 101^3.) The screenshot link is my very naive median-cut encoder, which is much worse than notcurses' octree encoder -- but that's still good enough that you might not be able to tell when I'm using iTerm2+PNG vs sixel.

Finally, and to my utter surprise, I may have encountered an actual xterm bug: https://gitlab.com/klamonte/jexer/-/issues/89 . Both foot and wezterm are doing quite well on sixel, and very nice options to have for testing.

All in all, I'm pretty happy with the possibilities from plain old sixel:

xterm_translucent_animations.mp4

[jmoore-pl]

I've got a patch on my fork that currently allows sixels to work. Basically, you can currently just dump a string containing an encoded sixel into the symbol field of a cell and the backend will totally output it just fine. Just need to add a field to cells that allows them to elect to be skipped by the Buffer::diff method when generating updates. That way, a sixel widget covering a given area can have all the cells under the sixel be skipped so it doesn't get drawn over by space characters.

To bring sixels onto the tui-rs side, we'll want a couple things, though:

• a sixel implementation (e.g. sixel-sys or a custom one if we don't like FFI)
• some way to reliably figure out terminal dimensions in pixels

We'll want the sixel implementation in order to convert image data into encoded sixel strings, if we want to abstract that much away from the user. We'll also want to know the pixel dimensions of a cell so that we can resize the sixel (and underlying image data) to fit its allotted area, or at least inform the user of the dimensions they need to fill. Otherwise, there's no way for us or them to make sure that the sixel actually occupies the space it was allotted.

It would probably be responsible to put this under an optional feature, due to the extra dependencies and non-universal support. That said, sixels do enjoy much wider support than other formats and conceptually very simple. We might consider supporting other protocols too, but I think sixels are a good starting point.

The top left is the cell with the sixel encoded. A, B, and C are the other corners. Notice the bottom line has no sixel. It's part of the drawable region (B and C are the actual bottom line of my emulator), but the sixel will get scrolled back up a line if it tries to draw there. Seems implementations prefer to add an extra newline after every sixel just to make sure there's no overlap. Not sure if there's any way around it.

[jmoore-pl]

Forgot to mention, I'm figuring out terminal dimensions by emitting an escape sequence (<ESC>[14t) and parsing stdin, but this isn't universal either. Neither termion or crossterm seem to support it, so this is a standing issue as well.

[ghost]

You can also use CSI 16/18t to estimate the cell dimensions (# pixels / # cells). But it's unusual for a TE to support only one of 14/16/18 and not all of them.

[ghost]

I found a trick to put sixel on the bottom row. It is standard, but some TEs don't do it right. https://gitlab.com/klamonte/jexer/-/issues/91

In practice you may be limited to bottom row cells that are within 1000x1000 pixels of the top left screen corner.

[jmoore-pl]

Awesome, looks like that did the trick! I'm looking at that octree sixel implementation now. For this project, I think it would be good to to have a native Rust implementation of sixel which could operate efficiently over a sparse set of image buffers. That way we can generate the screen-wide sixel we need in DECSDM needing to copy everything into a contiguous C-style buffer like we would for the sixel-sys bindings,

[ghost]

I've been horrible with polluting that thread with references so will stop doing that, but do check out chafa's implementation too (issue 27 "sixel support" over there, start at June 22, 2020). It might be more readable / Rust-able, and today on some (maybe many?) image types it is the fastest known encoder. But notcurses is working hard to beat it, so who knows where the outcome will be. :)