nim-notcurses: a wrapper for the Nim programming language

[michaelsbradleyjr]

It's a small start, but still a start:

https://github.com/michaelsbradleyjr/nim-notcurses

Things to note:

• The wrapper is about as raw it gets at this point. It will need some TLC, possibly another wrapper-layer and/or an API backed by the wrapper/s before it's nice to work with as a library for Nim.
• I've not yet published it as a package for nimble, but will do so once I'm convinced a good portion of the Notcurses API is reasonably usable via nim-notcurses. I decided to go ahead and add it, carpe diem: Added notcurses nim-lang/packages#1982.
• I've tagged v2.3.13, but I may delete and re-create that tag if I discover major problems with the wrapper as it exists currently.
• I need to add a GitHub Actions workflow so at least examples/hello_direct.nim is being compiled and run across Linux, macOS, and maybe even Windows.
• I need now to actually learn how to use Notcurses instead of dreaming about using it so I can (1) include some modestly interesting examples in the nim-notcurses repo and (2) consume it in a project that to-date is using (a fork of) nim-ncurses. To that end, I've been reading Notcurses' wiki, guide book, etc.
• Static linking should already be possible, but one has to know what to pass with e.g. Nim's compiler's --passL option. I tried earlier today for both libnotcurses.a and libnotcurses-core.a, but couldn't get it to work. I'll need to do more research, and once I know the "formula" I'll include some notes in the README (or an .md doc linked from it). There will, of course, be variances by platform, but I'll try to cover the bases.

[dankamongmen]

pkg-config --static --libs ought provide for your last issue:

[schwarzgerat](0) $ pkg-config --static --libs notcurses
-lnotcurses -lstdc++ -lnotcurses-core -lunistring -lm -ltinfo 
[schwarzgerat](0) $ pkg-config --static --libs notcurses-core
-lnotcurses-core -lunistring -lm -ltinfo 
[schwarzgerat](0) $ 

[WSLUser]

I hope the wrapper are added to this project once it seems ready for consuming. Since precedence has already been set by including other wrappers, it logically makes sense to consolidate them all to this repo.

[michaelsbradleyjr]

I understand the idea, but given how the nimble package manager works with repositories and how nimterop is configured to work in nim-notcurses, it would be a bit awkward.

The simplest thing is to keep it in a separate repo and have a link to it in this repo's README. I think it's still too early to add a link, though. I need to spend another weekend or two experimenting with it, and will probably add a not-so-raw wrapper on top of the raw one.

[michaelsbradleyjr]

Quick update:

For about half a month I've been working on an overhaul of nim-notcurses as personal time has allowed. The ongoing effort is taking place on the version_3_revamp branch. Some examples of usage of the Nim API are in examples/poc (cli1.nim is a little rough because I'm currently messing around 🙈 ).

It's still very much a WIP, there are miles to go, but I'm starting to be pleased with the overall shape the layers are taking.

I've updated the readme on that branch with respect to where/how I want to end up.

Highlights:

• Hand-rolled low-level wrapper around Notcurses' C ABI, as opposed to depending on nimterop to generate it (too much goes wrong).
• Higher-level "Nim oriented" API, which in the readme I refer to simply as the Nim API.

Notes:

• It's currently required that Notcurses already be installed, vs. the old behavior of its sources being downloaded and compiled on the fly. I eventually intend to restore the auto download-compile as an option, but I won't give attention to that until I'm finished with the API/wrapper and have ported all the examples and demo.
• Only v3 of Notcurses will be supported at first. I might one day backport my "overhaul effort" to support Notcurses v2 ... but maybe not.
• I've only been developing it on macOS and will eventually need to make adjustments re: other platforms. (works on Linux and macOS now, Windows is a WIP)
• I've only been developing it with Nim v1.6 and will eventually need to make adjustments re: older release lines of Nim I intend to fully support: v1.2 and v1.4.

The internal structure of my nim-notcurses overhaul (as opposed to the API it presents) is perhaps a bit unusual, but my chosen approach allows me a lot of flexibility re: wrapping and presenting core, CLI mode, and Direct mode with minimal code duplication and without dependence on compile-time options passed to the Nim compiler by users of nim-notcurses.

A side benefit of the approach is that I'm learning a lot more about C and Notcurses internals than I would have by relying purely on nimterop to generate the ABI wrapper.

---

I'll be opening some discussions and/or issues in the near future, because I'm already running into Notcurses things that aren't working as I (maybe wrongly) expect, and as far as I can tell they're behaviors of Notcurses and not my ABI wrapper nor API. I'll do my best to pose the questions in a way that doesn't require non-Nimmers to get hot and heavy with Nim, i.e. I'll try to frame them in terms of Notcurses' C headers and sources as much as possible.

Happy New Year 🎉

[dankamongmen]

i very strongly discourage effort in supporting notcurses 2. it's in the past, and as important as a snowflake.

[michaelsbradleyjr]

I've got GitHub Actions cruising along for my version_3_revamp branch:

https://github.com/michaelsbradleyjr/nim-notcurses/actions?query=workflow%3ATests+branch%3Aversion_3_revamp

Compilation and runs of my cli1, cli2, and cli3 are working on Linux and macOS now (per local testing), when building with Nim v1.2, v1.4, or v1.6. Granted nim-notcurses still doesn't have a lot of meat on its bones, but that's a bit of progress since my last update, and the stage is set now to plunge more deeply into porting the Notcurses examples and developing the API, etc. as time allows.

---

I'm having some trouble on Windows. It's in three parts, and I'll break it down here. If any of these seem like they're possible bugs or changes needed in Notcurses, let me know and I'll create issues for investigation. I'm also happy to try to create PRs, if/when I have the insight to do so.

1. Build files not copied to /ucrt64 dirs

Locally and in GHA, I'm building Notcurses on Windows like this in a clone of the repo:

$ cmake .. -G"MSYS Makefiles" -DCMAKE_BUILD_TYPE=Release -DUSE_DOCTEST=off -DUSE_PANDOC=off
$ make

When I run make install (requires msys2 shell to be opened as admin), the build files get arranged in C:\Program Files (x86)\notcurses.

I was a bit surprised they didn't get copied into the appropriate /ucrt64 dirs. So I worked out that, for now, I need to do:

$ cp -R ../include/notcurses /ucrt64/include
$ cp -R ../include/ncpp /ucrt64/include
$ cp *.dll /ucrt64/bin
$ cp *.dll.a /ucrt64/lib

With the files in place there, I can compile my cli1 and run it in an MSYS2 shell like this (DLLs are properly located at compile time and run time):

$ examples/poc/cli1

And in Windows Terminal like this:

set PATH=C:\msys64\ucrt64\bin;%PATH%
> cd C:\msys64\home\me\repos\nim-notcurses
> examples\poc\cli1.exe

What do you think? Could / should CMakeLists.txt be adjusted so that make install on Windows copies build files into the appropriate directories for the current MSYS2 environment? Maybe there's a way to avoid hardwiring C:\msys2\ucrt64, e.g. maybe someone is having luck with the CLANG32 or CLANG64 environment, or they installed the UCRT64 environment elsewhere.

2. dynlib problems

After sorting out (1) above, with only a few adjustments I got my cli1, etc. compiling on Windows right away. But I found that my programs would crash right away because they couldn't load/locate symbols from the DLLs at runtime, e.g. ncplane_set_scrolling.

When I did import notcurses/core instead of import notcurses and it "worked" (see 3 below), I realized that it has something to do with how Nim arranges for DLLs to be used:

From here:

proc ncplane_set_scrolling(n: ptr ncplane, scrollp: cuint): bool
  {.cdecl, dynlib: notcurses_lib, importc.}

notcurses_lib is libnotcurses.dll for import notcurses and it's libnotcurses-core.dll for import notcurses/core.

Nim's dynlib facility uses dlopen on mac/linux, while on windows it uses LoadLibraryA and GetProcAddress (I don't know much about it, just did some searching).

On linux and mac, it seems to be enough to tell Nim to inform the linker about both libnotcurses and libnotcurses-core, in addition to using dynlib.

But on Windows it seems that I need on a per symbol basis to know and specify if it's coming from notcurses or notcurses-core.

• Any ideas how on Windows/MSYS2 I can better inform the linker?
• If that's a dead end, how could I determine if a symbol would be coming from notcurses or notcurses-core?

It's possible to tell the Nim compiler to arrange for dlopen to not be used, i.e. to instead depend on linker flags. I tried that, but it didn't work: gcc started barfing about duplicated references to notcurses_init, etc.

The Notcurses binaries that are built on Windows with cmake/make (e.g. notcurses-input) don't have have this problem, so obviously there's shortcomings in the way the Nim compiler does things by default, with respect to what needs to be done for Notcurses on Windows.

3. mintty is spewing

My cli1 works perfectly in Windows Terminal, as well cli2 and cli3.

But in mintty all three nim-notcurses programs spew garbage into the terminal, and for cli1 I have to ctrl+c to end the program.

I'm using the -P flag with mintty and I have the Windows OS locale set to utf8. So something else is going on.

notcurses-input built with cmake/make doesn't have this problem in mintty, so again, there must be something related to how the sausage is getting made by the Nim compiler. I can probably sort it out eventually, but if someone has any ideas on where/how to investigate, I'll be grateful.

[dankamongmen]

i'm sorry not to have responded to this yet; i will do so this weekend!