-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
caching common tokens #821
Comments
I think on-demand interning will be enough, without pre-computation |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Our scanner copies the string contents of every token when creating it. But many tokens share the same source string:
public
,contract
,function
.We should calculate the most frequent tokens (using sanctuary datasets), and define them as static/const values in source, returning static references when possible, to eliminate allocations:
phf::Set::get_key
inToken::new()
to fetch the static refs and store them instead of cloning the original input, similar tosmol_str::SmolStr::new_static
.Token::new()
asconst
, storing the entire object instead of just the string in thephf
map.Rc<Token>
, so that all of them share the same instance/pointer.Once we have #808, we can evaluate the impact of these optimizations.
The text was updated successfully, but these errors were encountered: