Releases: belladoreai/llama3-tokenizer-js
Releases · belladoreai/llama3-tokenizer-js
v1.1.3
v1.1.2
Reorganize files (should not include functional changes)
v1.1.1
Fix TypeScript types
v1.1.0
Add parsing for special tokens so that they can be now inserted into normal text (without requiring any special function calls as before)