Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some manner of Sixel interface #200

Closed
dankamongmen opened this issue Dec 23, 2019 · 32 comments
Closed

Add some manner of Sixel interface #200

dankamongmen opened this issue Dec 23, 2019 · 32 comments
Assignees
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Milestone

Comments

@dankamongmen
Copy link
Owner

I'm not yet deeply knowledgeable regarding Sixel, but it seems pretty tight (where supported). It's definitely not a 1.0.0 thing, but we ought open up the Sixel floodgates sooner rather than later IMHO.

@dankamongmen dankamongmen added documentation Improvements or additions to documentation enhancement New feature or request question/research Further information is requested labels Dec 23, 2019
@dankamongmen
Copy link
Owner Author

libsixel looks pretty solid, and is MIT-licensed: https://github.com/dankamongmen/libsixel

so sixel seems sweet. Problem is lack of support -- it's not present in kitty, alacritty, or vte, at least. To get it working well in xterm, you need:

XTerm*decTerminalID: vt340
XTerm*numColorRegisters: 256

in your xresources database, or to launch it with:

xterm -ti vt340

(we can upgrade the number of color registers ourselves, ala lsix, so long as vt340 mode is used)

but yeah, that's just xterm. it does work over ssh, though.

@dankamongmen dankamongmen self-assigned this Feb 21, 2020
@dankamongmen
Copy link
Owner Author

And now, of course, it is obvious how we ought integrate Sixel: as a glyph blitting backend of ncvisual (see #622). Huzzah!

@dankamongmen
Copy link
Owner Author

it looks like we'll want to use sixel_encode()

/* convert pixels into sixel format and write it to output context */
SIXELAPI SIXELSTATUS
sixel_encode(
    unsigned char  /* in */ *pixels,     /* pixel bytes */
    int            /* in */  width,      /* image width */
    int            /* in */  height,     /* image height */
    int            /* in */  depth,      /* color depth: now unused */
    sixel_dither_t /* in */ *dither,     /* dither context */
    sixel_output_t /* in */ *context);   /* output context */

we'll need to provide an output context:

/* create output context object */
SIXELAPI SIXELSTATUS
sixel_output_new(
    sixel_output_t          /* out */ **output,     /* output object to be created */
    sixel_write_function    /* in */  fn_write,     /* callback for output sixel */
    void                    /* in */ *priv,         /* private data given as
                                                       3rd argument of fn_write */
    sixel_allocator_t       /* in */  *allocator);  /* allocator, null if you use
                                                       default allocator */

dankamongmen added a commit that referenced this issue May 27, 2020
@dankamongmen
Copy link
Owner Author

I spent some time looking through libsixel, and am unimpressed. It's lousy with scan-build errors and indeed even some basic compiler warnings. It also has a ton of crap we don't need. I'm thinking we just hand-write the NCBLIT_SIXEL blitter.

And we still don't seem to have a way to detect whether Sixel is available :/.

@dankamongmen
Copy link
Owner Author

lsix appears to be capable of determining this:

Error: Your terminal does not report having sixel graphics support.

Please use a sixel capable terminal, such as xterm -ti vt340, or
ask your terminal manufacturer to add sixel support.

You may test your terminal by viewing a single image, like so:

        convert  foo.jpg  -geometry 800x480  sixel:-

If your terminal actually does support sixel, please file a bug
report at http://github.com/hackerb9/lsix/issues

(Please mention device attribute codes: ^[[?6c)

@dankamongmen
Copy link
Owner Author

JFC, lsix is a bash script? heavens save us.

    # IS TERMINAL SIXEL CAPABLE?    # Send Device Attributes
    IFS=";" read -a REPLY -s -t 1 -d "c" -p $'\e[c' >&2
    for code in "${REPLY[@]}"; do
  if [[ $code == "4" ]]; then
      hassixel=yup
      break
  fi
    done

ugh, it looks like it requires interrogating the terminal :/

@dankamongmen
Copy link
Owner Author

Yeah, drop the libsixel crap. We'll proceed directly.

dankamongmen added a commit that referenced this issue Jun 5, 2020
@dankamongmen dankamongmen added this to the 1.7.0 milestone Jul 13, 2020
@dankamongmen
Copy link
Owner Author

The only terminals I can find with support for this are Xterm and mlterm (though maybe the OSX and Windows terminals have support; I'm unsure). Hrmm.

@dankamongmen
Copy link
Owner Author

We are now detecting Sixel support dynamically. If NCOPTION_DETECT_SIXEL is provided to notcurses_init(), we detect it upon startup. Otherwise, we detect it the first time (and only the first time) we attempt to use NCBLIT_SIXEL. If NCBLIT_SIXEL is provided together with NCVISUAL_OPTIONS_NODEGRADE, and Sixel is unavailable, the render attempt will fail.

dankamongmen added a commit that referenced this issue Jul 19, 2020
@dankamongmen
Copy link
Owner Author

Just stumbled across OCS1337, supported by hterm and iterm

dankamongmen added a commit that referenced this issue Jul 19, 2020
@dankamongmen
Copy link
Owner Author

OK, I've got all the detection and such working in dankamongmen/shitxel. I just don't see this being useful enough to finish out and commit at the moment. Perhaps we'll come back to it.

@dankamongmen dankamongmen removed this from the 1.7.0 milestone Jul 19, 2020
@dankamongmen
Copy link
Owner Author

Sixel Graphics
If xterm is configured as VT240, VT241, VT330, VT340 or VT382 using the
decTerminalID resource, it supports Sixel Graphics controls, a palleted
bitmap graphics system using sets of six vertical pixels as the basic
element.

CSI Ps c Send Device Attributes (Primary DA), xterm. xterm responds to
Send Device Attributes (Primary DA) with these additional
codes:
Ps = 4 ⇒ Sixel graphics.

CSI ? Pm h
Set Mode, xterm. xterm has these additional private Set Mode
values:
Ps = 8 0 ⇒ Sixel scrolling.
Ps = 1 0 7 0 ⇒ use private color registers for each
graphic.
Ps = 8 4 5 2 ⇒ Sixel scrolling leaves cursor to right of
graphic.

DCS Pa ; Pb ; Ph q Ps..Ps ST
Send SIXEL image, DEC graphics terminals, xterm. See:

         VT330/VT340 Programmer Reference Manual Volume 2:
         Graphics Programming
         Chapter 14 Graphics Programming

      The sixel data device control string has three positional
      parameters, following the q  with sixel data.
        Pa ⇒  pixel aspect ratio
        Pb ⇒  background color option
        Ph ⇒  horizontal grid size (ignored).
        Ps ⇒  sixel data

@dankamongmen
Copy link
Owner Author

CSI
? P i ; P a ; P v S
Set or request graphics attribute, xterm. If configured to support either Sixel Graphics or ReGIS
Graphics, xterm accepts a three-parameter control sequence, where P i , P a and P v are the item,
action and value:
Patch #350
8
2019/11/02XTerm Control Sequences
VT100 Mode
P i = 1 → item is number of color registers.
P i = 2 → item is Sixel graphics geometry (in pixels).
P i = 3 → item is ReGIS graphics geometry (in pixels).
P a = 1 → read attribute.
P a = 2 → reset to default.
P a = 3 → set to value in P v .
P a = 4 → read the maximum allowed value.
P v can be omitted except when setting (P a == 3 ).
P v = n ← A single integer is used for color registers.
P v = width ; height ← Tw o integers for graphics geometry.
xterm replies with a control sequence of the same form:
CSI
? P i ; P s ; P v S
where P s is the status:
P s = 0 ← success.
P s = 1 ← error in P i .
P s = 2 ← error in P a .
P s = 3 ← failure.
On success, P v represents the value read or set.
Notes:
• The current implementation allows reading the graphics sizes, but disallows modifying those
sizes because that is done once, using resource-values.
• Graphics geometry is not necessarily the same as “window size” (see the dtterm window manip-
ulation extensions). For example, xterm limits the maximum graphics geometry at compile time
(1000x1000 as of version 328) although the window size can be larger.
• While resizing a window will always change the current graphics geometry, the reverse is not
true. Setting graphics geometry does not affect the window size.

@joseluis
Copy link
Collaborator

joseluis commented Nov 2, 2020

I just tried running notcurses-demo on xterm -ti vt340 (or just xterm really) and the demos look very bad for me. Do they work well for you?

@dankamongmen
Copy link
Owner Author

Time to go to bed, but we're now generating some manner of pixel output through direct mode. We're outputting a bunch of sixel codes in rendered mode, but that ought be simple enough to resolve. Need to break down the colors; we're currently always providing 100;100;100, but it shouldn't be difficult to break down. We're close!

dankamongmen added a commit that referenced this issue Feb 28, 2021
dankamongmen added a commit that referenced this issue Feb 28, 2021
dankamongmen added a commit that referenced this issue Feb 28, 2021
dankamongmen added a commit that referenced this issue Feb 28, 2021
@dankamongmen
Copy link
Owner Author

I merged the first big bit of work -- all the infrastructure and documentation -- just now as #1372. Need to get sixel_blit() working properly now. Let's use lsix output and differential analysis to figure out what's going on there. lsix, btw, seems to happily use a pretty large number of color registers, certainly more than the 4 of the vt340, heh.

@dankamongmen
Copy link
Owner Author

dankamongmen commented Feb 28, 2021

getting closer...we can now draw a 20x20 monochromatic square, except that there are gaps between each sixel band, lol. but getting there! and we draw it in both direct and rendered mode.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants