Python Bindings

[nathaniel-may]

I noticed that almost all of the grammar repositories that provide bindings only provide Rust and Node. I'm looking to also provide Python bindings for the grammar I am building. How is this usually achieved?

[theHamsta]

Maybe you could add a setup.py and use language.build_library of https://github.com/tree-sitter/py-tree-sitter to build the parser. Since you only have to compile parser.c you could also use distutils to compile the library

[nathaniel-may]

Thanks! I did get something like this to work, I just didn't know if there was a preferred way to do this or if python consumers of grammars were reusing the provided rust or node bindings. It's also possible that there are just very few people working with tree-sitter in python 🤷‍♂️

[Freed-Wu]

I want to share my solution. I use Freed-Wu/tree-sitter-zathurarc@9ffe281 as an example, to show how to add python bindings. In this commit, only javascript binding bindings/node and rust binding bindings/rust is provided. I'll add a bindings/python. git clone the code in prepare stage.

1. Create bindings/python/tree_sitter_zathurarc/__init__.py:

from pathlib import Path
from tree_sitter import Language, Parser

__version__ = "0.0.1"
__version_tuple__ = (0, 0, 1, "", "")

language = Language(
    str(next((Path(__file__).parent / "data" / "lib").glob("zathurarc.*"))),
    "zathurarc",
)
parser = Parser()
parser.set_language(language)

2. Create pyproject.toml:

[build-system]
requires = ["setuptools", "tree-sitter"]
build-backend = "setuptools.build_meta"

# https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html
[project]
name = "tree-sitter-zathurarc"
description = "zathurarc grammar for tree-sitter"
readme = "README.md"
requires-python = ">= 3.10"
keywords = ["parsing", "incremental"]
classifiers = [
  "Development Status :: 3 - Alpha",
  "Intended Audience :: Developers",
  "Topic :: Software Development :: Build Tools",
  "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
  "Operating System :: Microsoft :: Windows",
  "Operating System :: POSIX",
  "Operating System :: Unix",
  "Operating System :: MacOS",
  "Programming Language :: Python :: 3 :: Only",
  "Programming Language :: Python :: 3",
  "Programming Language :: Python :: 3.10",
  "Programming Language :: Python :: 3.11",
  "Programming Language :: Python :: 3.12",
  "Programming Language :: Python :: Implementation :: CPython",
  "Programming Language :: Python :: Implementation :: PyPy",
]
version = "0.0.1"
dependencies = ["tree-sitter"]

[[project.authors]]
name = "Wu Zhenyu"
email = "[email protected]"

[project.license]
text = "GPL v3"

[project.urls]
Homepage = "https://tree-sitter-zathurarc.readthedocs.io"
Download = "https://github.com/Freed-Wu/tree-sitter-zathurarc/releases"
"Bug Report" = "https://github.com/Freed-Wu/tree-sitter-zathurarc/issues"
Source = "https://github.com/Freed-Wu/tree-sitter-zathurarc"

[tool.setuptools.packages.find]
where = ["bindings/python"]

[tool.setuptools.package-data]
tree_sitter_zathurarc = ["py.typed", "data/*"]

3. create setup.py:

#!/usr/bin/env python
r"""Setup
=========
"""
from setuptools import setup
from tree_sitter import Language

Language.build_library(
    "build/lib/tree_sitter_zathurarc/data/lib/zathurarc.so", ["."]
)

setup()

That's all. Just pyproject-build, you can get dist/tree_sitter_zathurarc-0.0.1-py3-none-any.whl.
pip install it and try from tree_sitter_zathurarc import parser! You can see Freed-Wu/tree-sitter-zathurarc@e8aaed5 to note these changes or git clone this commit.

In fact, the stages can be simplified, 3rd is unnecessary. We can move this step to upstream py-tree-sitter. I'll open a new discussion to discuss it.

And 1st and 2nd steps' file (a python and a pyproject.toml) can be generated automatically by tree-sitter generate. I just not sure their response about adding a bindings/python.

[Freed-Wu]

Discussion: tree-sitter/py-tree-sitter#178