SIMD

[Boshen]

Topic around using SIMD for performance.

[Boshen]

Whitespace skipping.

Backround reading:

• https://rapidjson.org/md_doc_internals.html#SkipwhitespaceWithSIMD
• https://github.com/Tencent/rapidjson/blob/bdc49ad80a3cff89a25839dcdcd6a68b457ec9f1/include/rapidjson/reader.h#L258-L517
• https://lemire.me/blog/2017/01/20/how-quickly-can-you-remove-spaces-from-a-string

[Boshen]

The results are promising from

• perf(lexer): use portable-SIMD to speed up whitespace scanning #26
• perf(lexer): use portable-SIMD to speed up multiline comment scanning #23

[strager]

In addition to lexing, I have considered using SIMD for line number calculations. I prototyped it a while ago (September 2020) but I didn't land it for some reason. quick-lint/quick-lint-js@c930a9f

SIMD can also be used for keyword recognition. (All keywords are 16 bytes or less.) I am currently making a YouTube video which explains this.

[Boshen]

Interesting! I can't wait for the video! I tried to understand multi-byte checking in SIMD but I just couldn't get my head around it.