Would it be possible for the blog plugin to setGlobalData({blogPosts})?

[ilg-ul]

With the current release code, I could not find a way to access a selection of the blog posts in standalone or docs pages.

Would it be possible for the blog plugin to save the blog posts globally (with setGlobalData()) and make the content available outside the blog plugin? Similar question for tags, authors, and possibly other data.

I saw a notice in the documentation (Global data is... global: its size affects the loading time of all pages of your site, so try to keep it small.), but I don't understand how the loading time of a static site can be affected by some JSON files saved during build time.

If saving the blogPosts in the blog plugin is not possible, is it reasonable to do this in the new processBlogPosts() hook? What solutions would be possible for the other data (tags, authors, etc)?

[Josh-Cena]

but I don't understand how the loading time of a static site can be affected by some JSON files saved during build time.

The global data is not just "saved as a JSON file". It is loaded globally in your site via const globalData = require("global-data.json") or something similar, so that anywhere in your site, you can use useGlobalData to retrieve this data. This JSON data directly lives in your main client bundle.

We most definitely do not want to save literally everything about blog posts globally. Nevertheless I think processBlogPosts could be a solution.

[ilg-ul]

We most definitely do not want to save literally everything about blog posts globally.

Why?

[Josh-Cena]

Because that means everything about your blog posts are loaded globally, even those that you don't use, and those that most people will never use.

[slorber]

I don't understand how the loading time of a static site can be affected by some JSON files saved during build time.

This is because you don't understand yet what React hydration is. Learn that and it will make sense.

---

Global data need to be loaded on every single page of your site before React could hydrate, because hydration assumes the data is loaded, because we use that data to display things such as the global site layout features (version dropdowns, i18n dropdowns etc).

If this data is very heavy, this delays the React hydration. Users on a bad network will encounter a bad UX where the site/page is visible, but it is not possible to interact with it for a few seconds. If users click on buttons, and the buttons do nothing, this is a really bad UX. Increasing global data size increase the time a site is presented as non-interactive, and should be avoided.

Now if you don't mind, you can extend the blog plugin contentLoaded to add the global data yourself. Just keep in mind that all this data needs to be loaded before the page becomes interactive, so it depends on the size you plan to put there.

#4138

const blogPluginExports = require("docusaurus-plugin-content-blog");

const defaultBlogPlugin = blogPluginExports.default;

async function blogPluginExtended(...pluginArgs) {
  const blogPluginInstance = await defaultBlogPlugin(...pluginArgs);
  const pluginOptions = pluginArgs[1];
  return {
    ...blogPluginInstance,
    contentLoaded: async function (params) {

      // Register yourself the extra global data you want here
      const { content, actions } = params;
      actions.setGlobalData(content);
      
      return blogPluginInstance.contentLoaded(params);
    },
  };
}

module.exports = {
  ...blogPluginExports,
  default: blogPluginExtended,
};

---

Another possibility is to have the blog plugin create additional pages, I've explained this setup here:
#4138 (comment)

This is more efficient (no global data usage) but will not work in all situations (particularly if you need data from 2 distinct plugins on the same page).

---

I doubt processBlogPosts would solve this problem and wonder what makes you think it could.

---

The solution we want to this "problem" (being able to inject any plugin data to anywhere) is React Server Components running at build time. But it's complicated to implement and we haven't started working on this yet.

#9089

Until we have this, I'll try to figure out if we can provide a temporary good enough workaround.

[slorber]

But I think that not providing a convenient way to share data between plugins at build time is a Docusaurus limitation.

We all agree on that, it's a known problem. Docusaurus plugins are kind-of sandboxed from one another.

I don't know, perhaps all my use cases are weird, being originated in how Jekyll works, with all data stored in global arrays.

Docusaurus is not Jekyll, because it's a SPA giving you a more stateful app and soft navigation instead of full page reloads:

• Jekyll only has to render HTML files at build time, and do not need to send this data to the frontend
• Docusaurus has to render HTML both at build time, and inside the browser when you click links, so the browser should download everything it needs to render the next page, in an efficient way. This usually gives a better UX, but is also more challenging to implement technically.

Gatsby used to add GraphQL queries alongside React components, letting you retrieve the data you need from a larger "graphql data lake". This looked like a good idea in 2015 when it came out, but it turns out it quickly became complicated and now devs do not like Gatsby because of that query layer.

import * as React from 'react'
import { graphql } from 'gatsby'

const HomePage = ({data}) => {
  return (
    <div>
     {data.site.siteMetadata.description}
    </div>
  )
}

export const query = graphql`
  query HomePageQuery {
    site {
      siteMetadata {
        description
      }
    }
  }
`

export default HomePage

Gatsby inspired the creation of Docusaurus v2, but the complex GraphQL layer that the community didn't like was dropped. Docusaurus is simpler but does not currently have a good data layer, to let you inject data from one plugin to another.

React Server Components are going to be the best solution to this problem, letting you "pick" the right data you need directly inside React components, using TS/JS object accesses instead of a query system.

---

I'm thinking of a possible good enough temporary solution.

The contentLoaded actually takes a 3rd undocumented arg allContent: https://docusaurus.io/docs/api/plugin-methods/lifecycle-apis#contentLoaded

async contentLoaded({actions: {createData, addRoute}, allContent})

Our debug plugin is currently able to create pages by reading content from other plugins:
https://github.com/facebook/docusaurus/blob/main/packages/docusaurus-plugin-debug/src/index.ts

This is undocumented because it's still considered internal, and I don't like this API, and the fact we have to "wait" for all plugin data to finally be able to call contentLoaded is not great either for optimizing execution times.

And this also prevents me from enabling even faster fine-grained hot reloads I'm working on in #9903
(not reloading whole site on change, but just a single affected plugin)

For these reasons I'm planning to introduce a new allContentLoaded lifecycle method asap. That would enable you to read from all the site data and create pages with it, or generating hardcoded json files that you could then read in React components directly through import myJson from @generated/myJsonFile

[ilg-ul]

For these reasons I'm planning to introduce a new allContentLoaded lifecycle method asap.

Great! Any ETE?

[slorber]

That's blocking me from completing #9903 properly (and it's the thing I'm working on atm) so very likely tomorrow 😄

[slorber]

Not sure it will perfectly resolve your problem (it's not the main motivation of this new lifecycle) but here's the PR: #9931

[ilg-ul]

Let me see if I understood it. For the historical site, I have to define a custom blog plugin, and implement the allContentLoaded() function where I get all blogPosts, I process them to compute the content for the chronology tables, I use createData() and addRoute() (as you explained in #4138 (comment)) to generate the home page?

Has the size of these JSON files generated by createData() any impact on site responsiveness? In other words, is it better to do all formatting before creating the data, or is it acceptable to pass some raw properties (that take some more space) and do the formatting in the component that displays the tables?

If I manage to do this, I guess that for the 20+ small sites it'll be easier to apply the same recipe (filter the blog posts, get those tagged with 'release' and use createData() and addRoute() to generate that specific page).

BTW, I did read the explanations in #9931 several times, and it is not clear for me why it was necessary to add allContentLoaded(), and not pass the same data as allContent via contentLoaded().

Also the text mentions that this new feature allows to aggregate data from multiple plugins, but it does not provide an example how to do it. (Currently I don't need this, the content of my chronology tables comes from a single blog plugin, but in my site I have 3 blog instances, and I might want to show data from the other blogs too).

[slorber]

Let me see if I understood it. For the historical site, I have to define a custom blog plugin, and implement the allContentLoaded() function where I get all blogPosts, I process them to compute the content for the chronology tables, I use createData() and addRoute() (as you explained in #4138 (comment)) to generate the home page?

You do not even have to extend the blog plugin to add your own allContentLoaded, since all plugins receive all the data, this can as well be a pure standalone plugin.

Has the size of these JSON files generated by createData() any impact on site responsiveness? In other words, is it better to do all formatting before creating the data, or is it acceptable to pass some raw properties (that take some more space) and do the formatting in the component that displays the tables?

It has impact only on the pages you inject that data into. If your homepage receives a lot of data, your homepage will be heave. Unlike global data this size will not impact ALL your pages, but just this one.

BTW, I did read the explanations in #9931 several times, and it is not clear for me why it was necessary to add allContentLoaded(), and not pass the same data as allContent via contentLoaded().

That is related to the DX optimizations I'm working on as part of #9903. If contentLoaded received that data, then on any file edit, we'd have to run this lifecycle for ALL the plugins. What we want instead is that if you edit a blog post, we only have to call the affected blog instance contentLoaded() method, and we do not have to call the docs contentLoaded(). This method creates files and can be expensive so we should try to only call it when needed. Most plugins do not need to read content from one another. If you do not understand this part it's not a problem but it will enable me to give a much faster hot reload experience on file edits, that would otherwise be unsafe.

Also the text mentions that this new feature allows to aggregate data from multiple plugins, but it does not provide an example how to do it. (Currently I don't need this, the content of my chronology tables comes from a single blog plugin, but in my site I have 3 blog instances, and I might want to show data from the other blogs too).

Agree, here's a full example that I'll also add to the PR:

https://stackblitz.com/edit/github-agngt8?file=MyHomePage%2FMyHomePagePlugin.js,MyHomePage%2FMyHomePage.js,docusaurus.config.js

export default async function MyHomePagePlugin(context, options) {
  return {
    name: 'my-home-page-plugin',
    async allContentLoaded({ allContent, actions }) {
      const allBlogs = allContent['docusaurus-plugin-content-blog'];
      const blog1 = allBlogs['blog1'];
      const blog2 = allBlogs['blog2'];
      const allBlogPosts = [...blog1.blogPosts, ...blog2.blogPosts];

      actions.addRoute({
        path: '/',
        exact: true,
        component: '@site/MyHomePage/MyHomePage.js',
        modules: {
          allBlogPosts: await actions.createData(
            'allBlogPosts.json', // The file name doesn't matter, we'll simplify this API soon
            JSON.stringify(allBlogPosts)
          ),
        },
      });
    },
  };
}

export default function Home(props) {
  const { allBlogPosts } = props;
  return (
      <main>
        <h2>All blog posts</h2>
        <ul>
          {allBlogPosts.map((blogPost, i) => (
            <li key={i}>{blogPost.metadata.title}</li>
          ))}
        </ul>
      </main>
  );
}

[ilg-ul]

That was helpful, thank you!