Render images directly from the terminal

[neverping]

Back in time, terminals were always text-based.

Currently, almost everyone uses a terminal on a GUI environment. It amazes me to not having a terminal that doesn't render a image if I do a "cat sunset.jpg".

It would be nice if Rio could detect such output and renders it. Or maybe offering a built-in command to display them, maybe "draw image.jpg".

[raphamorim]

That would be a nice to have for after the first release. Marking it as wish list.

Believe kitty terminal also does that with icat: https://sw.kovidgoyal.net/kitty/kittens/icat/

[chase]

The basic protocol for images in Kitty is pretty simple:
https://sw.kovidgoyal.net/kitty/graphics-protocol/

I used it to make a simple but fully functional Chromium-based web browser that renders in Kitty and WezTerm (with Kitty protocols enabled): https://github.com/chase/awrit

There is also the iTerm2 image protocol which has support in a lot of newer terminal emulators: https://iterm2.com/documentation-images.html

[AnonymouX47]

The basic protocol for images in Kitty is pretty simple:

Point of correction... it's not that "simple" on the end of the terminal emulator.

It's probably the main reason why no terminal emulator other than Kitty implements nearly half of the protocol [correctly].

[chase]

The basic protocol for images in Kitty is pretty simple:

Point of correction... it's not that "simple" on the end of the terminal emulator.

It's probably the main reason why no terminal emulator other than Kitty implements nearly half of the protocol (correctly).

Fair enough, certainly the protocol in its entirety is complicated.

The scope of this issue isn't to support image IDs, arbitrary positions, animation etc. It's to render images directly in the terminal, and that can be done with a subset of the protocol, described within the first few paragraphs.

A minimal subset, similar to iTerm2's protocol, can be implemented: permitting only PNG filenames to be transmitted and displayed immediately at the current cursor position. Then handling a clear screen to remove said image.

Although you certainly have more experience with such things than myself, I wouldn't want the maintainer to get discouraged. Incremental improvement and all that.

[AnonymouX47]

The scope of this issue isn't to support image IDs, arbitrary positions, animation etc. It's to render images directly in the terminal, and that can be done with a subset of the protocol, described within the first few paragraphs.

Defining that "subset" is where the problem begins.

A minimal subset, similar to iTerm2's protocol, can be implemented: permitting only PNG filenames to be transmitted and displayed immediately at the current cursor position. Then handling a clear screen to remove said image.

Although you certainly have more experience with such things than myself, I wouldn't want the maintainer to get discouraged. Incremental improvement and all that.

From the little experience I have with the two protocols mentioned here, I'll advice the iTerm2 protocol be implemented first (because it's a lot less complex than Kitty's) and the Kitty graphics protocol should only be implemented when the project is more mature and developers/contributors are ready to take it on properly.

For some insight, check out the current state of support in Wezterm (just a search through the issues on there will do). By the way, the support is declared experimental but comparing it to the state of the iTerm2 protocol support (and how long each has been in development) makes my point clear.

Please Note: The above mention/reference is not intended to talk lightly/negatively about the project or its author/developers (in fact, Wezterm is my daily driver and it's doing very well in almost every other aspect I can think of).

The summary of all I've said is... It's better to have a [almost] complete, spec-compliant implementation of a less-capable protocol than a highly incomplete implementation of a more-capable protocol. Also, it'll be less headache for the developers.

---

That said, there's also sixels (requested in #46) but that has its own unique set of headaches, primarily due to the lack of a complete central/standard specification and limited access to a reference implementation (in fact, the closest to a reference implementation I've come across is probably not something any relevant implementation today would want to follow head-on).

Anyways, sixels are probably still simpler to implement than the kitty graphics protocol given the limited scope.

I should note that my experience with sixels is currently limited, there are others way more knowledgeable/experienced on the topic.

[AnonymouX47]

Hello @raphamorim !

I noticed some sixel implementation work was underway, though, I'm not sure of the state as I'm yet to test.

I just wanted to inform you about an ongoing related discussion that's quite important as I believe it's important that things are done well and correctly right from the onset.

See hpjansson/chafa#192, you might wanna follow along and possibly chime in.

Well done and thank you.