Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] feat: offline-only export #2802

Draft
wants to merge 15 commits into
base: main
Choose a base branch
from
Draft

Conversation

pankgeorg
Copy link
Collaborator

@pankgeorg pankgeorg commented Feb 6, 2024

will fix: #2771 (eventually)

[WIP]: this is an attempt to bundle all the assets required for a static export to work completely offline, with all the assets bundled in the HTML.

Parcel is supposed to do this, but there are quite a few caveats that make this not work as-is.

Current state:

creates a 17MB HTML file with all the assets bundled, but

  1. parcel is not smart enough to bundle some of the images
  2. javascript bundling is also messing up some imports and some MIME types

Needs more work

Copy link
Contributor

github-actions bot commented Feb 6, 2024

Try this Pull Request!

Open Julia and type:

julia> import Pkg
julia> Pkg.activate(temp=true)
julia> Pkg.add(url="https://github.com/fonsp/Pluto.jl", rev="pg/self-contained-export")
julia> using Pluto

@fonsp
Copy link
Owner

fonsp commented Feb 6, 2024

Interesting! Curious if the bundler issues can be fixed easily!

Did you look into external tools that could do this? (i.e. make an offline-ready snapshot version of this html file.) I tried the "Save complete page" feature from Chrome but that did not work. :(

And did you consider solutions that involve a custom CDN? Maybe you could self-host? You need to statically serve the frontend-dist folder of every Pluto version. Or maybe cdn.jsdelivr.net could be allow-listed?

The solution could also be a Julia script: julia viewnotebook.jl path_to_my_nb.html which starts a static file server and shows the notebook.

@pankgeorg
Copy link
Collaborator Author

Interesting! Curious if the bundler issues can be fixed easily!

me too! some issues seem like noone cares to fix

Did you look into external tools that could do this? (i.e. make an offline-ready snapshot version of this html file.) I tried the "Save complete page" feature from Chrome but that did not work. :(

I've tried parcel with various configurations, monolith and inliner. They all get stuck somewhere in the dynamically loaded images, in loading the modules despite them being text/javascript or some combination

And did you consider solutions that involve a custom CDN? Maybe you could self-host? You need to statically serve the frontend-dist folder of every Pluto version. Or maybe cdn.jsdelivr.net could be allow-listed?

No this is not an option for our use case, unfortunately.

The solution could also be a Julia script: julia viewnotebook.jl path_to_my_nb.html which starts a static file server and shows the notebook.

This might be good to do either way. I'll make a small package that does this, it can be a nice workaround for now.

@pankgeorg
Copy link
Collaborator Author

And, of course, there is the service worker approach

@fonsp
Copy link
Owner

fonsp commented Feb 7, 2024

One problem with the SW is that it only works the second time a page is loaded, but you can work around it by triggering a refresh on the first load. You should check that this will work on file:// though.

@pankgeorg
Copy link
Collaborator Author

pankgeorg commented Feb 7, 2024

One problem with the SW is that it only works the second time a page is loaded, but you can work around it by triggering a refresh on the first load.

Yes, that was my thought too

You should check that this will work on file:// though.

MDN says it should work

> Locally-delivered resources such as those with http://127.0.0.1 URLs, http://localhost and http://*.localhost URLs (e.g. http://dev.whatever.localhost/), and file:// URLs are also considered to have been delivered securely.

But will know more when I have a MWE

Well, it doesn't 🤦🏾w3c/ServiceWorker#578

@fonsp
Copy link
Owner

fonsp commented Apr 16, 2024

Got it! Some changes I had to make:

url in CSS

For the url(https://...svg) in CSS: use two plugins:

"transformers": {
        "*.svg": [
            "...",
            "@parcel/transformer-inline"
        ]
    },
    "optimizers": {
        "*.svg": [
            "...",
            "@parcel/optimizer-data-url"
        ]
    }

And also, I had to learn to remove the frontend/.parcel-cache dir when changing the parcel config, otherwise your changes have no effect.

url in JS

For cases where we were using new URL("https://...", import.meta.url) to tell parcel to bundle these files, I could not find an easy solution to make this work, because parcel uses import.meta.url in the bundle to make this work, but it breaks when loading the JS from a base64 url.

To solve this, I make add these URLs as <link elements to editor.html and use document.head.querySelector(...).href to get the value from code. This lets parcel bundle the files and Julia can base64 them, and it will still work.

fonts

This I still need to figure out!

@fonsp
Copy link
Owner

fonsp commented Apr 17, 2024

Also got mathjax and the fonts :)

@fonsp
Copy link
Owner

fonsp commented Apr 17, 2024

With the JS changes and inlining ionicon svgs, the exports work offline, with mathjax, only the fonts don't load.

But inlining the svgs bring the bundled editor.css from 80kB to 120kB and that's acceptable! So we can do this in the main bundle, and we don't need a separate bundle for offline support.

I'm thinking of tweaking our font stacks a bit to make the fallback fonts look more like Alegreya and Vollkorn, and just keeping it like that! Then the offline html will be just 1.5MB with slightly different looking fonts when viewing without internet. But we don't need a second bundle :)

@fonsp
Copy link
Owner

fonsp commented Apr 17, 2024

#2896 to make the fallback fonts look nicer :) which is something i wanted to do for a long time!

@sjkelly
Copy link
Contributor

sjkelly commented Jun 3, 2024

Is there any chance this can be merged as-is? This seems like it is useful for procedurally generating notebook pages to HTML even without a UI:

using Pluto

s = Pluto.ServerSession()

nb_file = "plotly_static.jl"

nb = Pluto.SessionActions.open(s, nb_file; run_async=false)

# Generate the HTML file
html_content = Pluto.generate_html(nb)

# Write the HTML content to a file
open("notebook.html", "w") do io
    write(io, html_content)
end

@fonsp
Copy link
Owner

fonsp commented Jun 4, 2024

Hey @sjkelly !

I also like this PR! Btw the code you wrote already works on the current Pluto release :) but the generated HTML does not work offline.

I want to enable this feature by default for all exports, so that our HTML files will always work without relying on the jsdelivr.com CDN like we currently do.

What's left for this PR is #2898. Without that, the generated HTML files will not use Pluto's standard typefaces but always use one of the default system fonts. And that's not nice, because we want a what-you-see-is-what-you-get experience.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature request: HTML Notebook Archive
3 participants