Add ApexChart based visualizations

[sydp]

Adds ApexChart based visualizations to frontend-ng

Checks

• All tests succeed.
• Unit tests added.
• e2e tests added.
• Documentation updated.

Closing issues

Put closes #XXXX in your comment to auto-close the issue that your PR fixes
(if such).

cast.webm

[berggren]

Hey @sydp This is a big PR :) It will take some time to review. I'll catch up with you offline to see if we can break it up a bit.

[rocketeeer]

bump

[jkppr]

Alright, given the size of the PR I have to split the review work into multiple sessions. So this is part 1/X. I'll cover the files in the Visualization folder in part 2.

[jkppr]

(2/X) I continued with some of the chart files. Please take a look at the comments below.

[jkppr]

The minimum settings we need to render an aggregation we need field, aggregation type and then some information depending on the type (e.g. max values and chart type).
Would it be a lot of effort to trigger & auto-update the preview on the right side when the minimum information are provided or changed? (Instead of the user clicking the reload button)

[sydp]

I am somewhat reluctant to auto update since this will trigger an aggregation on the server end which could be expensive/slow, depending on the selected field/aggregation type (and users may unintentionally select wrong values)

Not sure what the effort, I don't imagine it would be "too much" (famous last words) but I'll hold off for now until you confirm this is what you would like to happen.

[jkppr]

Thanks, I get your concerns with auto-update. The current behaviour is some mix: If I generate and change the chart, it will reload automatically. I guess this is due to changing the chart just being in the frontend and not re-running the aggregation on the backend?

Let's go with a load/reload button for now and keep the behaviour the same. So no auto change for switching charts for example. We can move to auto-update later when we have some metrics on how people are using the feature.

What do you think?

[sydp]

I guess this is due to changing the chart just being in the frontend and not re-running the aggregation on the backend?

Yes, that's correct - the Load button is only for retrieving the aggregated results so the user can preview them in the selected chart (and update in the front-end) before deciding to save/clear/cancel.

I'm fine with leaving as keeping the behaviour as is, but of course equally happy to tweak if desired.

[jkppr]

3/X

[jkppr]

The BarChart has some truncation for the field names. But the Column chart and LineChart don't do the truncation. See screenshots:

vs

Is this something that can be fixed easily or something for a future PR?

[sydp]

It might be difficult to control when labels can be unexpectedly large. Not an ideal "fix" but the user can simply turn off x-axis labels and hover over bar/data point to see the value (similar to the histogram chart in the explore view).

[sydp]

For info, details on the behaviour of labels from ApexCharts:

https://apexcharts.com/docs/multiline-text-and-line-breaks-in-axes-labels/

[jkppr]

I ran some small user-tests with colleagues in the office and one common feedback was that it is unclear what the aggregation options mean and how they work.
What do you think about using a combination of title and subtitle for each entry? Where the subtitle explains what this aggregation actually does. Similar to this mock for the DFIQ implementation:

[sydp]

by aggregation options, do you mean the aggregators themselves or the options for each aggregator?

[jkppr]

Sorry, I'm talking about the meaning of "Auto Time Interval", "Rare terms", "Time interval" and "Top K terms".

[sydp]

got it thanks for clarifying.

I'll work on adding some information but these were essentially aliases for the following aggregations

"auto time interval" - https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-autodatehistogram-aggregation.html#search-aggregations-bucket-autodatehistogram-aggregation
"rare terms" - https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-rare-terms-aggregation.html
"time interval" - https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-datehistogram-aggregation.html#search-aggregations-bucket-datehistogram-aggregation
"top k terms" - https://opensearch.org/docs/1.3/aggregations/bucket/terms/

I can revert to just using the aggregator name 1:1 if this causes confusion.

[sydp]

#3040 (comment)

I'm not able to comment to this in thread but this seems to be working as intended on my machine. perhaps it could be stale data after changing the max document count value. does clicking the "load button" produce the expected data?

edit: disregard, I think i am able to replicate. seems the value isn't propagating.