Lua: plugins can easily get Lua annotations metadata

[justinmk]

Problem

For clients to get function signatures and type information, currently we need to model APIs in C. It's not easy for clients to generate their own language-specific interface to the Nvim Lua stdlib (vim.*, :help lua-stdlib).

Expected behavior

We have started to annotate the Lua stdlidb with LuaLS EmmyLua type hints. We could use this to generate something like :help api-metadata and expose it as a secondary tree via nvim_get_api_info() and nvim --api. Or alternatively, just ship a luameta.lua or luameta.mpack file in the runtime files, which can be loaded by clients to generate their wrappers.

• ship foo.mpack files (currently undocumented but they live in runtime/doc/*.mpack)
• ship LuaLS annotations in runtime/lua/vim/_meta for quasi-introsopection. #19918
• ship util function that makes it easy for plugins to consume LuaLS annotations.

---

• Benefits
  • Clearly answers the question of where new functionality should go:
    • API (libnvim) = expose low-level things that are otherwise limited to internal usage.
    • Lua stdblib = high-level functionality that builds on top of the API.
    • Related: Prepare for version 1.0 #20451
• Costs
  • Lua is less natural to use for C clients linking to libnvim.
  • Eventually, this would imply that the Lua stdlib has a strong contract like :help api-contract, since this would mean that RPC clients now depend on the Lua stdlib. Thus it would be harder to evolve the Lua stdlib.
    • Mitigation: mechanism for marking Lua functions as "experimental" (or just "mental" ;).

[lewis6991]

Now we have fully populated runtime/lua/vim/_meta, is this ticket now resolved?

[justinmk]

Almost. I think we need a util function that makes it easy for plugins to consume the LuaLS annotations. The annotations are the "data structure" that we ship, but AFAIK it's not easy to get a Lua structure from that.

So let's use this issue to track that idea.

Now we have fully populated runtime/lua/vim/_meta

We are still missing Ex commands I think? Also normal mode.

[lewis6991]

Out of curiosity, what are some potential use cases for plugins to consume this type of data?

[justinmk]

Use-cases:

• which-key plugins which dynamically display and search command names+docs
• fuzzy-finders for the same purpose
• could also be used as a starting point for replacing gen_vimdoc.py

Alternatively, we could require LSP for these things. But I'm hesitant to close the door on this idea; a treesitter-based util function doesn't seem like wasted effort to avoid a permanent LSP dependency, and it could start out fairly primitive (no markdown support).