feat: allow inlined rules to match empty

[rooney]

Fix #3220

[amaanq]

Tokens (terminals) can't be inlined - the linked issue doesn't have an example, is this for the case where a rule is an optional (thus can match nothing) but is inlined? So wouldn't these rules containing this inline need to be checked as well then? If rule a is comprised of a single rule b, and rule b is an inlined rule matching an optional, then rule a also becomes invalid since it can match an empty string.

[maxbrunsfeld]

So wouldn't these rules containing this inline need to be checked as well then? If rule a is comprised of a single rule b, and rule b is an inlined rule matching an optional, then rule an also becomes invalid since it can match an empty string.

Yeah, I think you are right @amaanq. @rooney if we allow inline rules to match the empty string, we'll need additional logic to correctly check that other rules (which use inline rules) don't match the empty string due to inlined rules. Is the complexity worth it, or is the existing behavior fine?

[rooney]

Tokens (terminals) can't be inlined - the linked issue doesn't have an example, is this for the case where a rule is an optional (thus can match nothing) but is inlined?

Yes -- more specifically (in real-world scenario): some sequence of optionals that ultimately can match empty, that in turn be used as a common subrule by other (non empty-matching) rules.

So wouldn't these rules containing this inline need to be checked as well then? If rule a is comprised of a single rule b, and rule b is an inlined rule matching an optional, then rule an also becomes invalid since it can match an empty string.

Yeah, I think you are right @amaanq. @rooney if we allow inline rules to match the empty string, we'll need additional logic to correctly check that other rules (which use inline rules) don't match the empty string due to inlined rules. Is the complexity worth it, or is the existing behavior fine?

Right, ok, I'll try to find a way that handles the transitivity in a non-complex way.

Btw, current behavior (before this patch) can actually match empty string just fine, e.g:

  rules: {
    source_file: ($) => seq("foo", $.blank, "bar"),
    blank: () => "",
  },

(not sure if this is intentional or not)