What happened to the docusaurus integration?

[osseonews]

Can't find the docusaurus integration anymore when logging into cloud. if we just scrape the website running docusaurus, the results are not good. Are you no longer integrating with docusaurus?

This your image in your docs for setting up docusaurus, but this setting is no longer available in cloud?

https://docs.askorama.ai/oss/docussarus-2.png

[faustoq]

We just re-activated the Docusaurus integration. We are currently testing the scraper, would you please provide more details on what should be improved in your opinion? Thanks for the feedback.

[osseonews]

Scraping caused alot of duplicate content, same doc appear many times, with / without . I don't want to bother with a complictaed exclusion formula, but wouldn't even know where to start if I even know how to do it. Plus there is no control at all in terms of what content is getting placed in the index. I might be missing something, but i couldn't even actually tell what was in the index. I think scraping is going to be extremely difficult to do. Offers little control.

[osseonews]

BTW, the problem with the docusaurus plugin is that it doesn't read MDX files correctly. So for example, in our MDX file we have a component, , which is used to display questions and answers. The plugin correctly scrapes this section, but for the content it includes the actual component, instead of the content that the component generates. Not sure how to get around that. If we could edit the index in some way online, we could put in the proper content, or maybe there is some way to do this programatically.

[osseonews]

I'm guessing we can just download the code for docusaurus plugin and write our own customizations in the parseHTMLContent function so that if there is an component there, we can just generate the actual content needed for that component dynamically.

[osseonews]

Just want to add that your docusaurus plugin is actually quite incredible, though I think we have to customize the code a bit for our needs (seems quite easy to do, though). In literally 5 minutes, we had a hybrid search set up on our site and in then in another 30 minutes, using your secure proxy, and vector search, we had a functional AI driven app on our website which interfaced with our docs. Not sure how many docusaurus websites there are out there, but they can all use this. So I wouldn't bother with the scraping for a docusaurus website, your plugin is much, much better and easier to control, as far as content.