Bootstrap AST and parser

[tyranron]

Moved from cucumber-rs/cucumber#153
Part of cucumber-rs/cucumber#124

Synopsis

This is the first PR in the story of Cucumber expressions support.

Solution

This PR implements Cucumber expressions AST and a parser for it. Next PRs will be adding AST into regex transformation and support inside proc-macros.

Checklist

• Created PR:
  • In draft mode
  • Name contains Draft: prefix
  • Name contains issue reference
  • Has assignee
• Documentation is updated (if required)
• Tests are updated (if required)
• Changes conform code style
• CHANGELOG entry is added (if required)
• FCM (final commit message) is posted
  • and approved
• Review is completed and changes are approved
• Before merge:
  • Milestone is set
  • PR's name and description are correct and up-to-date
  • Draft: prefix is removed
  • All temporary labels are removed

[tyranron]

This deviates from the original grammar by missing the parameter variant.

Need to discuss this on voice.

[ilslv]

@tyranron there is a Note: section which says

While parameter is allowed to appear as part of alternative and option in the AST, such an AST is not a valid a Cucumber Expression.

Basically ARCHITECTURE.md describes AST that tries to be wider than Cucumber Expression for some reason (the only I can think of is to make parser implementation simpler and then error on conversion to real Cucumber Expression).
But in reality it doesn't make much sense to me. Especially alternative definition

alternative         = optional | parameter | text
text                = whitespace | ")" | "}" | .

This grammar suggests that alternative may have unescaped whitespaces, which is not true: example. They have to be escaped at least to avoid ambiguity.

[ilslv]

It's not even EBNF, as I understand, as wikipedia says that repetition is described with {...} and not with (...)*. It looks more like regex, but still has , for concatenation.

[tyranron]

@ilslv would you be so kind to make a PR to upstream that adjusts the described grammar to be accurate and precise enough. Because it really bothers: having spec which doesn't reflect reality, while implementations don't follow the spec 😕

[tyranron]

@ilslv

It's not even EBNF, as I understand, as wikipedia says that repetition is described with {...} and not with (...)*. It looks more like regex, but still has , for concatenation.

From your link: * is a repetition, and ( ... ) is grouping. So we have a group repetion here. I don't see any mistakes in that. And in Markdown it has ```ebnf notation 🤷‍♂️

[tyranron]

It's better to preserve here the original EBNF grammar syntax, so it will match the architecture document, and preserve original naming asap.

Markdown can do EBNF pretty well, though:

parameter = "{" (name | "\" name_to_escape)* "}"

[ilslv]

As described above, provided EBNF grammar tries to describe language that is wider than Cucumber Expression, but has errors. Grammar in my PR describes exactly Cucumber Expression.

[tyranron]

ack @ilslv

As I've re-created this PR and represent its author, GitHub refuses to let me review it. So, please, just ping me for the next review.

[ilslv]

@tyranron can you add me as collaborator in this repo, so I can push AST into Regex transformation branch?

[tyranron]

@ilslv here you go

[ilslv]

@tyranron for some reason i still can't push a new branch

16:18:17.613: [cucumber-expr] git -c credential.helper= -c core.quotepath=false -c log.showSignature=false push --progress --porcelain origin refs/heads/ast-to-regex-transformation:ast-to-regex-transformation --set-upstream
remote: Permission to cucumber-rs/cucumber-expressions.git denied to ilslv.
fatal: unable to access 'https://github.com/cucumber-rs/cucumber-expressions.git/': The requested URL returned error: 403

[ilslv]

@tyranron should I prepare this PR for review or send all at once in #2?

[tyranron]

@ilslv let's merge them separately.

[ilslv]

FCM

Cucumber expression AST and parser (#1)

[ilslv]

@tyranron ready for review

[tyranron]

@ilslv thanks! Quite a brilliant code! 💪