Cannot trigger conflict in ambiguous grammar OR How to specify a "fallback rule"

[wldmr]

I'm trying to write a grammar for Inkle's Ink language for interactive fiction. Here's my test case:

{blofeld and old: I saw that old git,|Dunno,} I said.

{blofeld is old: and that's a fact,|I already told you about his age,} I said.

These two lines do different things:

• The first one is "conditional text" – if the expression blofeld and old is true, then show "I saw …", else show "Dunno,".
• The second one is an "alternatives" construct: The first time the interpreter gets here, show "blofeld is old: and that's a fact", the second time show "I already …".

The distinguishing factor between these two cases is that blohfeld and old can be parsed as an expression, while blofeld is old can not (because "is" is not an operator).

I thought I could resolve this by declaring a conflict, essentially saying "try both, then pick the survivor". But tree-sitter just tells me "Warning: unnecessary conflicts", so I'm clearly not understanding something important here. Is there any way to get this to work?

Minimal Grammar

module.exports = grammar({
  name: 'ink',

  inline: $ => [$._expr],

  conflicts: $ => [
    // If I enable any of these, tree-sitter tells me 'unnecessary conflicts' 🤷
    [$.alternatives, $.conditional_text],
    [$._expr, $.text],
    [$.identifier, $.text],
  ],

  rules: {
    ink: $ => seq(repeat($._line)),
    _line: $ =>  seq($.flow, alias(/\n/, '\\n')),

    // positive or negative precedence here determines if the flow rule tries conditional_text or alternatives,
    // but it doesn't get it right in context.
    text: _ => token(prec(-10, /[^\n\{\}|]+/)),

    flow: $ => prec.right(repeat1(choice(
      $.conditional_text,
      $.alternatives,
      $.text,
    ))),

    alternatives: $ => seq(
      '{',
      repeat1(choice('|', $.flow)),
      '}',
    ),

    conditional_text: $ => seq(
      '{',
      field('condition', $._expr),
      ":",
      field('iftrue', $.flow),
      "|",
      field('else', $.flow),
      '}',
    ),

    _expr: $ => choice(
      $.identifier,
      $.binary,
    ),

    binary: $ => prec.left(seq($._expr, field('op', $._binary_operator), $._expr)),

    _binary_operator: _ => choice("and", "or"),

    identifier: _ => /[a-zA-z_][a-zA-Z0-9_]*/,

  },

});

Parse Trees

Massaging the grammar (by commenting out $.conditional_text or playing with the precedence of the $.text token), I can get either of the syntax kinds to match, but not both at the same time:

and

What I want is the left branch to have the conditional_text shape, while the right branch should match the alternatives branch.

---

Bonus Question: While I'd love to understand what I'm missing here, the underlying problem seems to be that the whole language basically works like "try to match a control structure and if that fails, it's text". Is there a pattern in tree-sitter to implement such an "ultimate fallback rule"?

[wldmr]

OK, I've cracked it (I think):

IDENT_REGEX = /[a-zA-z_][a-zA-Z0-9_]*/

module.exports = grammar({
  // …
  conflicts: $ => [
    [$.identifier, $.text],
  ],

  rules: {
    // …
    text: $ => prec.right(repeat1(choice(
    IDENT_REGEX,  // this needs to come first, because the next regex is more general and tree-sitter goes by rule order. If the next regex matched an identifier, it wouldn't trigger the conflict.
      /[^\s\{\}|:]+/,
      ':',
      $._binary_operator,
    ))),
    // …
    identifier: _ => IDENT_REGEX,
  },
});

My mistakes:

1. I had (stupidly) pre-decided the conflicts by giving text a lexical precedence (token(prec(…))) for the text.
2. Text tokenized differently from _.expr and the alternatives syntaxes (for instance, ':' needs to be its own token).
3. All the expression tokens must be repeated as text tokens.

The first two are pretty obvious. The third one is sort of understandable, but the specifics are a bit mystifying: I can reference $._binary_operator in text, but not $.identifier, or it doesn't trigger the conflict. That is non-obvious to me. If anybody could shed any light on this, I'd appreciate it.

Anyway; I think I now understand the basics of it. Let's see if I can make it work with the actual Ink syntax (which is of course more complicated than this minimal example 🤯).