New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1 没有uer/utils/data.py 文件 2 使用BERT-WWM对整词进行遮罩，如何添加自定义领域词典？ #359

Open

943433536 opened this issue Apr 12, 2023 · 0 comments

943433536 commented Apr 12, 2023

readme文件里说可以通过修改 uer/utils/data.py 中的代码将分词工具由jieba替换为其他分词工具。但是没有 uer/utils/data.py 这个文件，我在 uer/utils/mask.py文件里找到了import jieba，请问修改mask文件是否正确？
是否可以直接加上一句jieba.load_userdict()实现添加自定义词典？还需要对google_zh_vocab.txt进行修改吗？

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment