Add new splitter to process QA type file(now only support JSON) and add Toggle button in knowledge_base page #3298
+80
−16
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I wrote a new splitter to improve the processing of QA-type knowledge(Now only supports JSON, as shown in the example). I also added a Toggle button on the knowledge_base page to switch between the QA splitter and the normal splitter (ChineseRecursiveTextSplitter defined in kb_config.py).
I created a PR because I noticed that many people are encountering the same issue (#3164, #893, and others).
Here are the updated page and test results for the QA splitter: