-
Notifications
You must be signed in to change notification settings - Fork 64
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Skipping whitespace tokens #156
Comments
const moo = require('moo')
const lex = moo.compile({
ws: {match: /\p{White_Space}+/u, lineBreaks: true},
word: /\p{XID_Start}\p{XID_Continue}*/u,
op: moo.fallback,
})
;[...lex.reset('while ( a < 3 ) { a += 1; }')]
.filter(t => t.type !== 'ws')
.map(t => t.value) |
@nathan The documentation doesn't describe this feature: does it need to be updated? |
The documentation needs to be updated to document As for the rest, I think Nathan's just demonstrating that since a moo lexer object is an Iterator, you can use |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is it possible to skip tokens when defining a lexer?
I want to split a string into a list of tokens without whitespace, but I don't know if Moo can do this:
Input string:
"while ( a < 3 ) { a += 1; }"
List of tokens:
["while","(","a","<","3",")","{","a","+=","1",",";","}"]
The text was updated successfully, but these errors were encountered: