Page tables load very slow in 4.2

[timkinali]

Description

After updating to 4.2, some page sections with layout: table load extremely slow. 100 pages ~10 seconds, 30 pages ~6 seconds. The issue goes away when changing layout to list, cardlets or cards. Confirmed by other users too.

In the forum it was suggested by @rasteiner that the toggle field preview fix in this commit 75eb89c can be the cause:
"It seems to introduce a new instantiation (new Form($model)) which is run for every row in the table."

Your setup

Kirby Version
Kirby 4.2

Additional context
More details in the forum post https://forum.getkirby.com/t/page-sections-load-slow-kirby-4-2-issue/31373/3

[tobimori]

interesting! noticed the same with dreamform submissions since they use the table layout.

[distantnative]

Tricky one... Indeed before 4.2 the rows weren't processed by the Form class. But that produced wrong values, no defaults etc. – we'll have to see if/how we can make it more performant while using the Form class.

[rasteiner]

PS: I haven't like bisected it or anything, mine was just guess by looking at what changed with table layouts between 4.1 and 4.2.
Maybe the problem's somewhere else.

Anyway if it really were the new Form($model)...
10s/100items = 0.1s/item
That's more than expected for a simple "new Form" (even if that triggers loading the blueprint), so I guess there is some O(n²) recursion going on somewhere.

[distantnative]

I haven't looked closer at it either yet. But I'd also think this is the main think that changed regarding table values.

[rasteiner]

Looked at this again, but it's actually not so easy to reproduce (like: I personally didn't get it to reproduce).

I've created a test setup with the worst things I could imagine, but a pages section with layout: table and limit: 100 still loads in ~70ms on my rather slow test server (php 8.2 in debian docker image on ubuntu) and 4.2.0.

here's the blueprints I've used:
Parent:

title: Test many children
sections:
  pages:
    layout: table
    limit: 100
    columns:
      title: true
      toggle: true

Child:

title: Child

sections:
  files:
    type: files
    layout: cards
  pages:
    type: pages
    layout: table
    parent: page.parent
    limit: 100
  fields:
    type: fields
    fields:
      toggle:
        label: Checkbox
        type: toggle
      info:
        text: the website has {{ site.index.count() }} pages

Could you share the blueprints of the parent and child pages? Maybe also some information on the server where this problem shows?

[gbdesign2023]

Could you share the blueprints of the parent and child pages?

In my case, ALL children and grandchildren are read out. Of course, I cannot post any blueprints of this. I have posted the blueprint with the table in the forum:
https://forum.getkirby.com/t/page-sections-load-slow-kirby-4-2-issue/31373/10

It feels like the display takes twice as long on the iPhone as on the desktop PC.

[georgobermayrsaf]

+1 form my side. Have the same issue with pages tables sections. Its not related the the columns I display. The load is way slower in table layout than in one of the other layouts. In my current setups loads take more than 30s for such a section, vs 70ms for a non-table section.

[rasteiner]

+1 form my side. Have the same issue with pages tables sections. Its not related the the columns I display. The load is way slower in table layout than in one of the other layouts. In my current setups loads take more than 30s for such a section, vs 70ms for a non-table section.

could you share the blueprints of both the pages section as well as the page blueprint of the pages being displayed in it?

[timkinali]

Here's a video... You can see tables loading slow while lists and cards load fast. The table pagination is also slow.
https://www.dropbox.com/scl/fi/lkrtspfguygmiv6ytv9jh/Sk-rminspelning-2024-05-08-kl.-13.30.00.mov?rlkey=k6grulvztglaj9qyuqg2u0b83&st=25dkctjh&dl=0

I also disabled all plugins with no change.

[distantnative]

@timkinali Could you please still share your blueprints? That would help reproducing and finding the issue (and hopefully a solution).

[georgobermayrsaf]

could you share the blueprints of both the pages section as well as the page blueprint of the pages being displayed in it?

The section blueprint is rather simple:

label: Subpages
type: pages
templates:
  - storytelling
  - product
  - website.academy.entrypoint
  - website.events.entrypoint
  - website.journal.entrypoint
  - website.jobs.entrypoint
  - website.news.entrypoint
  - website.pressreleases.entrypoint
  - website.webinars.entrypoint
  - loginarea.entrypoint
  - search.results
layout: table
search: true
columns:
  template:
    label: Template
    value: '{{ page.intendedTemplate }}'
    type: tags
  path:
    label: Path
    value: '{{ page.websiteId }}'
  lastUpdate:
    label: Last Update
    value: '{{ page.modified("d/m/Y H:i") }}'
query: page.index(true)
sortBy: title asc
image:
  query: page.teaser.image.toFile
  ratio: 1/1
  cover: true

But the issue is the same with just:

label: Subpages
type: pages
layout: table

I'm afraid I can't share the page blueprint. The setup is too complex with too many extended fields, block field sets and so on. There are all kinds of field types in the blueprint.

[Akasiel]

I have the same issue. On my local machine it takes 1.6m to load a page table section and in a test environment 51.92s.

[distantnative]

@Akasiel Any chance you can share the full blueprints?

[georgobermayrsaf]

I did a bit more digging today. I still can't provide a blueprint, but it seems the main performance issue are blocks fields with custom block types. In my setup I have a lot of them, some of the custom block types even have nested blocks with again custom block types in it. Once I remove the blocks field from the blueprint loading is very fast as in older versions. But if I add one custom block type after another loading gradually gets worse.

[georgobermayrsaf]

BTW: The move pages dialog seems to have similar issues. Loading of a subpage tree with a lot of subpages (lets say 100) takes very long. From what I gathered in the code, here also the blueprint is completely evaluated. I was also able to pin it down to blocks fields in the blueprint that cause these performance issues.

[afbora]

I had an idea. Let me explain with an example:

Imagine there are 1000 note pages under the notes page. When loading the section, the Form class is re-instantiated for each note page. Within this class the page blueprint (note.yml) is loaded. This is done for all 1000 pages. The main reason it is slow in my opinion is that it loads the blueprint 1000 times. If we cache the page blueprint somewhere (in static variable or something like that), only 1 blueprint will be loaded for 1000 pages. This means a very serious performance increase.

What do you think? @getkirby/backend

[timkinali]

@timkinali Could you please still share your blueprints? That would help reproducing and finding the issue (and hopefully a solution).

@distantnative Sorry for not replying. Like others my blueprints are a bit complex with extends and so forth. I'm happy to share my whole blueprint folder -- or even the entire site -- but not publically. I can however send it to you (or someone else in the Kirby team). Let me know how to go about it!

[distantnative]

@timkinali if you could send them to [email protected] that would be appreciated. Thanks!