Can we find a way to collaborate on tree-sitter support?

[mattmassicotte]

First, huge congratulations! This is totally awesome, and shipping is a big deal.

Thanks for the SwiftTreeSitter shout-out in the acknowledgements, that's very kind of you! Do you think we could ever find a way to collaborate in that area? Seems like there's so much overlapping effort here. Perhaps https://github.com/krzyzanowskim/tree-sitter-xcframework might be a starting point?

(If you have interest in sharing why you went your own way on tree-sitter bindings, I'd love to hear. But, everyone's allowed to do their own thing and there's definitely no need justify!)

[mattmassicotte]

So here's a twist I was not expecting!

I was inspired by your Package.swift to try out adding SPM support to tree-sitter directly. It's possible! And, so while I was able to open a PR, I found that someone had beat me to it! tree-sitter/tree-sitter#1736

I also experimenting with doing the same thing for parsers. I've tried just a few so far, and while it is more complex, it is also possible! Everything via SPM just seems like a best-case option, so I'm moving SwiftTreeSitter over to that.

Just wanted to keep you up to date, since there have been so many developments in the last day or so.

[simonbs]

First, huge congratulations! This is totally awesome, and shipping is a big deal.

Thanks for the kind words!

[...] Do you think we could ever find a way to collaborate in that area? [...]

(If you have interest in sharing why you went your own way on tree-sitter bindings, I'd love to hear. But, everyone's allowed to do their own thing and there's definitely no need justify!)

It would be great to find a way to collaborate on bridging Tree-sitter to Swift! Ideally there should be no need for us to maintain two not-so-trivial libraries that solves the same exact problem. So why did I write my own library in the first place then? Here are the main reasons:

1. The most important reason is that I needed to get familiar with Tree-sitter's API. There are many ways to do that but I found that the best way for me to understand the API's, how they're intended to be used, and their limitations were to start interfacing with them.
2. When setting out to work with Tree-sitter from Swift, I naively thought that I'd only need a very narrow subset of Tree-sitter's API bridged. Therefore I thought it would be unnecessary to add a dependency. While the entire API isn't bridged in Runestone's codebase, there's certainly more than I thought I originally expected.
3. Along the lines of point no. 2, I enjoyed the idea of having a very minimal set of dependencies in Runestone. Right now Runestone is only depending on Tree-sitter.

Point 3 still stands. I'd like to go far to avoid dependencies but that's not to say that I don't want any, and I think the Tree-sitter bridge may be a good candidate for a dependency.

I haven't had a close look at SwiftTreeSitter in a while, so I'm unsure if it would be more or less a drop-in replacement of Runestone's current implementation, or if any changes are needed, or if it in fact makes sense to come up with some alternative.

I was inspired by your Package.swift to try out adding SPM support to tree-sitter directly. It's possible! And, so while I was able to open a PR, I found that someone had beat me to it! tree-sitter/tree-sitter#1736

Yep, Runestone is just wrapping Tree-sitter in a Swift package. I never thought of adding it as a PR to the Tree-sitter repository though. That would be great to have!

I also experimenting with doing the same thing for parsers. I've tried just a few so far, and while it is more complex, it is also possible!

That would be amazing! I'd love to know how that goes. I'm far from happy with my current solution for adding parsers to Runestone.

[mattmassicotte]

This all makes total sense. I'm super into small libraries, so that results in more dependencies. But I know that isn't always popular. I'm really happy we're talking about it. But you definitely need to make sure you're building something that makes sense and feels right to you!

I was able to get 5 parsers packaged for SPM so far (Go, GoMod, Swift, JSON, and Ruby). The swift parser (perhaps ironically) was the most complex, because it does not check in the generated C code. I haven't yet opened PRs for them, but will this week. I've bundled the queries in the packages too. They don't even depend on the runtime, so they can all be totally independent. It's quite nice!