Replies: 5 comments
-
We plan to do some performance comparisons between our canvas and webgl backends so that we can provide guidance on which is more appropriate under particular circumstances. No doubt we will document our benchmarking code and analysis so that others can repeat it if they wish as the results will be system dependent. But they are internal comparisons within Bokeh. External comparisons with other libraries needs to be done by someone without affiliation to any of the compared libraries if it is to be taken seriously. So if such a comparison was to be done I am sure someone here would provide help on the Bokeh side of things, but I think you are unlikely to find anyone here to instigate such a comparison. |
Beta Was this translation helpful? Give feedback.
-
It would be great to develop some benchmarks to run in CI continuously. And just speaking for myself, I would not be opposed at all to mechanically reporting such results. I might be even be inclined to casually tweet them out at times. But I agree with @ianthomas23 that I would not personally be interested at all in conducting comparisons across different libraries for marketing purposes, for all the reasons stated. cc @bokeh/dev |
Beta Was this translation helpful? Give feedback.
-
At least something to show version by version improvements along with where you stand w.r.t. others. |
Beta Was this translation helpful? Give feedback.
-
As someone who maintains a library that supports Bokeh, Plotly, and Matplotlib and whose output can be deployed using Voila, Bokeh Server, or Dash, I'd love to have some solid comparisons about the speed of each of these libraries in certain key applications, so that we could talk to our users about tradeoffs between different libraries. Each library has strengths and relative weaknesses, and it's difficult for me to be confident about those, let alone individual users trying to choose a library. Plus if one library is dramatically worse than another on a particular important case, it can help spur that library to improve in response. I think the ideal way to get such believable benchmarks is to have them run by someone external who is equally skilled at each library's usage, along with opportunities for each library's maintainers to analyze what the performance comparison code is doing and point out when the software is not being used most efficiently or effectively. But I've never seen that case arise in practice, because almost no one has the opportunity and incentive to become equally skilled in multiple such libraries. So I'm doubtful that the ideal case is likely enough to matter to anyone. With that in mind, I personally would be happy to see any reasonably well-done benchmarks comparing any important application across multiple libraries. Such benchmarks will never be perfect, and can easily be misused, but they would still be useful to me! |
Beta Was this translation helpful? Give feedback.
-
Official Matplotlib benchmarks are at https://github.com/matplotlib/mpl-bench |
Beta Was this translation helpful? Give feedback.
-
Currently performance comparison between different versions of Bokeh and other libraries is lacking.
With the introduction of BokehJS and WebGL rendering can such benchmarks be added. Perhaps a starting point can be: https://github.com/leeoniya/uPlot/tree/master/bench but expanded.
This should test for handing:
Beta Was this translation helpful? Give feedback.
All reactions