Cannot parse multi catch statement when catch keyword written after \n

[QthCN]

I'm doing some work based on tree-sitter-kotlin .

When I parse something which include multi catch statement and catch keyword written after \n, tree-sitter just parse them as call expression, like catch():

fun main() = try {
        } catch(e: E1) {

        }
        // new line here
        catch(e: E3) {
        }

output:

➜  tree-sitter-kotlin git:(master) ✗ tree-sitter parse example-file
(source_file [0, 0] - [7, 0]
  (function_declaration [0, 0] - [3, 9]
    (simple_identifier [0, 4] - [0, 8])
    (function_body [0, 11] - [3, 9]
      (try_expression [0, 13] - [3, 9]
        (catch_block [1, 10] - [3, 9]
          (simple_identifier [1, 16] - [1, 17])
          (user_type [1, 19] - [1, 21]
            (type_identifier [1, 19] - [1, 21]))))))
  (comment [4, 8] - [4, 24])
  (call_expression [5, 8] - [6, 9]
    (call_expression [5, 8] - [5, 20]
      (simple_identifier [5, 8] - [5, 13])
      (call_suffix [5, 13] - [5, 20]
        (value_arguments [5, 13] - [5, 20]
          (value_argument [5, 14] - [5, 15]
            (simple_identifier [5, 14] - [5, 15]))
          (ERROR [5, 15] - [5, 19]
            (ERROR [5, 17] - [5, 18])
            (integer_literal [5, 18] - [5, 19])))))
    (call_suffix [5, 21] - [6, 9]
      (annotated_lambda [5, 21] - [6, 9]
        (lambda_literal [5, 21] - [6, 9])))))
example-file	0 ms	(ERROR [5, 15] - [5, 19])

But if I write catch just after }, everything is ok:

fun main() = try {
        } catch(e: E1) {

        }/* without newline*/ catch(e: E3) {
        }

output:

➜  tree-sitter-kotlin git:(master) ✗ tree-sitter parse example-file
(source_file [0, 0] - [5, 0]
  (function_declaration [0, 0] - [4, 9]
    (simple_identifier [0, 4] - [0, 8])
    (function_body [0, 11] - [4, 9]
      (try_expression [0, 13] - [4, 9]
        (catch_block [1, 10] - [3, 9]
          (simple_identifier [1, 16] - [1, 17])
          (user_type [1, 19] - [1, 21]
            (type_identifier [1, 19] - [1, 21])))
        (comment [3, 9] - [3, 29])
        (catch_block [3, 30] - [4, 9]
          (simple_identifier [3, 36] - [3, 37])
          (user_type [3, 39] - [3, 41]
            (type_identifier [3, 39] - [3, 41])))))))

I have already try the methods that mentioned in conflicting-tokens, but none of them can work.

Currently I can only make parser work correctly if add \n/\r/\t/ after catch_block.

                catch_block: $ => seq(
                        "catch",
                        "(",
                        repeat($.annotation),
                        $.simple_identifier,
                        ":",
                        $._type,
                        ")",
                        $._block,
                        // it looks like that tree-sitter will `eat` these tokens
                        optional(repeat(choice(
                            "\n",
                            "\r",
                            "\t",
                            " "
                        )))
                ),

It just looks like tree-sitter will eat these tokens, so I guess it might be an issue related to tree-sitter instead of tree-sitter-kotlin.

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label bug to this issue, with a confidence of 0.60. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[maxbrunsfeld]

I moved this to the Discussion board. We are trying to move away from using GH issues for these sorts of questions.

[maxbrunsfeld]

This isn't a bug in Tree-sitter. It is correct to give you a \n token when it sees a newline after the catch block, because a \n token is valid in that position.

I'm not very familiar with kotlin, or its statement-termination rules. Is there any simple rule for determining when a newline should be interpreted as a statement-terminator? Allowing an optional \n between catch blocks might be a fine solution. Alternatively, you may need to define an external scanner that applies some arbitrary C code for deciding whether or not to create a \n token. That is what tree-sitter-javascript does, for example.

[QthCN]

Thank you for the response, I'll learn how tree-sitter-javascript implement it.