Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pre_seg_text开头加入判断是否全是特殊字符，并防止text为空字符，更改多个换行符的替换的代码 #962

Open
wants to merge 2 commits into
base: fast_inference_
Choose a base branch
from
Open

pre_seg_text开头加入判断是否全是特殊字符，并防止text为空字符，更改多个换行符的替换的代码 #962

wants to merge 2 commits into from

Commits on Apr 11, 2024

  1. 更改pre_seg_text中开头以及对多个换行符的替换 

    在text = text.strip("\n") 后面加入判断，防止text为空字符串。
使用正则式来替换原来将\n\n换为\n的式子
    @jmaple12
    jmaple12 committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    0dd1d38 View commit details
    Browse the repository at this point in the history

  2. 在pre_seg_text中加入判断，若text中都是特殊字符，则将其记为空字符串，防止后面复读。

    @jmaple12
    jmaple12 committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    d09022e View commit details
    Browse the repository at this point in the history