Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handling of quota exceeded errors in workbox-precaching #1312

Closed
jeffposnick opened this issue Feb 14, 2018 · 13 comments
Closed

Handling of quota exceeded errors in workbox-precaching #1312

jeffposnick opened this issue Feb 14, 2018 · 13 comments

Comments

@jeffposnick
Copy link
Contributor

Library Affected:
workbox-precaching

This is closely related to #1308, but the exact analog is GoogleChromeLabs/sw-precache#345

Let's leave runtime caching aside for this issue, and assume that there's a web app instance that has revision N of their assets precached. The web app is updated and deployed, and now there's an attempt to download and cache a few of the new/updated assets that make up revision N+1. workbox-precaching does this in the install handler, and will fail installation if any of the individual cache.put() operations fail. (Since all the assets from revision N are still cached at this point, it's not too hard to imagine scenarios in which caching the new assets brings you over quota.)

From the point of view of the web page clients, there are events fired on the associated service worker registration showing the service worker progressing from the installing state to the redundant state. Unless (somehow) quota is freed up at some point, that installing -> redundant transition will happen indefinitely, leaving the user stuck revision N of all the precached assets indefinitely.

We should do something to help developers out here. What about a pattern like:

if ('serviceWorker' in navigator) {
  window.addEventListener('load', function() {
    navigator.serviceWorker.register('/sw.js').then(function(registration) {
      registration.addEventListener('updatefound', function() {
        var newWorker = registration.installing;
        newWorker.addEventListener('statechange', function() {
          if (newWorker.state === 'redundant') {
            if ('storage' in navigator && 'estimate' in navigator.storage) {
              navigator.storage.estimate().then(function(results) {
                var percentUsed = (results.usage / results.quota) * 100;
                // Let's assume that if we're using 95% of our quota, then this failure
                // was due to quota exceeded errors.
                // TODO: Hardcoding a threshold stinks.
                if (percentUsed >= 0.95) {
                  // Get rid of the existing SW so that we're not stuck
                  // with the previously cached content.
                  registration.unregister();

                  // Let's assume that we have some way of doing this without inadvertantly
                  // blowing away storage being used on the origin by something other than
                  // our service worker.
                  // I don't think that the Clear-Site-Data: header helps here, unfortunately.
                  deleteAllCachesAndIBD();
                }
              });
            } else {
              // What about browsers that don't support navigator.storage.estimate()?
              // There's no way of guessing why the service worker is redundant.
            }
          }
        });
      });
    });
  });
}

and then encapsulate all that logic in the hypothetical new module described in #1129?

@jeffposnick
Copy link
Contributor Author

Thinking out load some more: we might be able to address this inside of the workbox-precaching install handler, by detecting cache.put() quota errors there, and using it as a signal to unregister the current service worker and clear all caches.

That would have the benefit of not requiring developers to change the way they register their service workers, and we would more context about which caches/IDB entries need to be deleted if the cleanup happened inside of our service worker.

@gauntface
Copy link

  1. (Self Service) We could clear the runtime cache and retry. If still fails, clear cache and unregister the service worker. This would fallback to the network and sw would start failing on install as normal.
  2. (Dev Option) We offer a cb function and on failing precache, developer can do anything they want (including message page - clear cache - clear something else, .....)
  3. Default to self service and dev cb results in replacing default behaviour

@raejin
Copy link

raejin commented Apr 12, 2018

@jeffposnick In the case of Safari 11.1, it doesn't support the navigator.storage API atm. What are some of the other possibilities that will fall in the newWorker.state === 'redundant' statement being true?

I'm trying to tackle this problem in general regardless of the browser. The plan is to delete all the known caches generated by our service worker.

@tgangso
Copy link

tgangso commented Jun 11, 2018

Would like to see a solution for this!

@raejin
Copy link

raejin commented Jun 11, 2018

@tgangso Relevant pull request for a solution #1505

@tgangso
Copy link

tgangso commented Jun 11, 2018

Very interesting, thanks @raejin

@srosset81
Copy link

@jeffposnick Since we've updated from Workbox 2 to 3, we have Quota exceeded errors when the SW is updating, which makes the update sometimes fail.
I've used the purgeOnQuotaError option on all runtime caching but it doesn't solve the problem, as indeed you write in #1505 :

If you're precaching a massive amount of data, but not runtime caching much, this PR is not likely to help.

What are the solutions ? In v2 it worked fine, maybe because there was no -temp cache during the SW installation ?

@jeffposnick
Copy link
Contributor Author

There's not a specific solution in place for Workbox v3.

You'd have to trim down the amount/size of URLs that you're precaching.

Coming up with an automated solution is going to require more thought, and is not something that we've been able to address yet.

@budarin
Copy link

budarin commented Oct 10, 2021

@jeffposnick
is there any solution to this issue?

I am currently using the code:

navigator.serviceWorker
.register(swPath})
.then((registration) => {
    registration.addEventListener('updatefound', () => {
        const installing = registration.installing;
    
        if (installing) {
            const onInstallingStateChange = () => {
                if (installing.state === 'redundant') {
                    // redirect
                    // or alert about quote error
                }
    
                if (installing.state === 'installed') {
                    installing.removeEventListener('statechange', onInstallingStateChange);
                    resolve(registration);
                }
            };
    
            installing.addEventListener('statechange', onInstallingStateChange);
        }
    });
})
.then((registration) => {
   // start app
})

@jeffposnick
Copy link
Contributor Author

Hello @budarin—there's no one-size-fits all solution here, so we've kept this open to track possibilities.

Listening for an installing service worker entering the redundant stage is a valid way of detecting it from the main page.

Alternatively, if you wanted to perform some asynchronous action from inside your your service worker when there's a quota error, you can use registerQuotaErrorCallback() from workbox-core to automatically run your code. This function could potentially purge some runtime caches, clear out IndexedDB, or even call self.registration.unregister(), depending on what makes the most sense for your use case.

In the future, we may expose a cacheWriteDidFail strategy lifecycle event (#1530), allowing you to add a custom plugin to the strategy used by workbox-precaching, and take specific action when a given cache write fails.

@budarin
Copy link

budarin commented Oct 11, 2021

Hi @jeffposnick !

Is there any workbox precache method or callback params method to clear pre cache in case of quota exceeded error?

@jeffposnick
Copy link
Contributor Author

Clearing the cache can be done with the standard Cache Storage API methods:

import {cacheNames} from 'workbox-core';

// Call this as your custom quota error callback, or anywhere else:
async function deletePrecache() {
  await caches.delete(cacheNames.precache);
}

Note that the next time the service worker installs, it's going to retry caching all the items in the precache manifest. So you might just end up redownloading everything and then running into quota issues again, unless the user also frees up additional space on the device or unless you reduce the size of your precache manifest.

@tomayac
Copy link
Member

tomayac commented Apr 25, 2024

Hi there,

Workbox is moving to a new engineering team within Google. As part of this move, we're declaring a partial bug bankruptcy to allow the new team to start fresh. We realize this isn't optimal, but realistically, this is the only way we see it working. For transparency, here're the criteria we applied:

Thanks, and we hope for your understanding!
The Workbox team

@tomayac tomayac closed this as completed Apr 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants