Sentence piece unigrams vs BPE for citrinet-like models #2129
jprobichaud
started this conversation in
General
Replies: 1 comment 1 reply
-
Good question ! We use WPE for Librispeech models, but unigram for most of the Nemo checkpoint releases. We did experiment with SPE BPE, and found no significant difference in WER for Librispeech compared to either unigram or WPE. However these weren't extensive analysis so there is a margin for noise. Overall, it would be interesting to know if SPE unigram does better than BPE for a particular dataset / case. |
Beta Was this translation helpful? Give feedback.
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I'm curious what people think makes more sense: for CTC-based models, like Citrinet, would sentence piece BPE tokenizer be better than unigram tokenizers?
I have some evidence that suggests that unigram tokenizers would fly well with attention decoders but make the life of a CTC decoder much harder.
Anyone as notice that too? Any potential justification for this?
Beta Was this translation helpful? Give feedback.
All reactions