Querying at different parts of the tree gives inconsistent results

[didroe]

I'm trying to run a query at a non-root node and it isn't matching. When I run the same query at the root it works correctly.

This seems to have something to do with the (Ruby) grammar containing a hidden rule. Due to the inconsistency, I assume this is an issue with Tree Sitter itself, rather than the grammar.

Example code:

#include <string.h>
#include <stdio.h>
#include <tree_sitter/api.h>

TSLanguage *tree_sitter_ruby();

int main() {
  TSParser *parser = ts_parser_new();
  ts_parser_set_language(parser, tree_sitter_ruby());

  const char *source = "def m\nsome.call\nend";

  TSTree *tree = ts_parser_parse_string(parser, NULL, source, strlen(source));

  uint32_t query_error_offset;
  TSQueryError query_error_type;
  TSQuery *query = ts_query_new(
    tree_sitter_ruby(),
    "(call)",
    6,
    &query_error_offset,
    &query_error_type
  );

  if (query_error_type != TSQueryErrorNone) {
    printf("query error: %d", query_error_type);
    return 1;
  }

  TSQueryCursor *query_cursor = ts_query_cursor_new();
  TSQueryMatch match;

  // Execute query from the root
  TSNode root_node = ts_tree_root_node(tree);
  ts_query_cursor_exec(query_cursor, query, root_node);
  bool found_from_root = ts_query_cursor_next_match(query_cursor, &match);
  printf("root string: %s\n", ts_node_string(root_node));
  printf("found from root: %d\n", (int)found_from_root);

  // Execute query from the call node
  TSNode call_node = ts_node_child(ts_node_child(ts_node_child(root_node, 0), 2), 0);
  ts_query_cursor_exec(query_cursor, query, call_node);
  bool found_from_call = ts_query_cursor_next_match(query_cursor, &match);
  printf("call string: %s\n", ts_node_string(call_node));
  printf("call type: %s\n", ts_node_type(call_node));
  printf("found from call: %d\n", (int)found_from_call);

  return 0;
}

Expected output:

root string: (program (method name: (identifier) body: (body_statement (call receiver: (identifier) method: (identifier)))))
found from root: 1
call string: (call receiver: (identifier) method: (identifier))
call type: call
found from call: 1

Actual output:

root string: (program (method name: (identifier) body: (body_statement (call receiver: (identifier) method: (identifier)))))
found from root: 1
call string: ("_call") receiver: (identifier) method: (identifier)
call type: call
found from call: 0

[mikeHTweety]

+1 on this problem

[vjerci]

It's impacting me too.

[ahelwer]

When you explore the tree with the cursor you can reach nodes that aren't printed out when you print the tree. Can you print out all the child nodes you can programmatically find from the parent of the node from which you execute the second query? So all the node's siblings.

[didroe]

I added the following code:

  TSNode call_parent = ts_node_parent(call_node);
  printf("parent: %s\n", ts_node_string(call_parent));
  for (int i = 0; i < ts_node_child_count(call_parent); i++) {
    printf("node %d: %s\n", i, ts_node_string(ts_node_child(call_parent, i)));
  }

And it outputs only this:

parent: (body_statement (call receiver: (identifier) method: (identifier)))
child 0: ("_call") receiver: (identifier) method: (identifier)

[cfabianski]

@ahelwer I feel like it could be coming from this location

tree-sitter/lib/src/subtree.c

Line 880 in d1d8193

cursor += snprintf(*writer, limit, "(\"%s\")", symbol_name);

Haven't been able to test it yet though :(

[amaanq]

seems like this was fixed in #3191, sorry for not noticing earlier