Pending batches expiring in 5 months, not 30 days

[kwugirl]

Ruby version: 3.0.6p216
Rails version: 6.1.7.7
Sidekiq / Pro / Enterprise version(s): 7.2.0 / 7.2.0 / 7.2.0

Hello! The wiki says that "Pending batches will expire in Redis after 30 days" but we are seeing batches created 1 week ago showing in the admin UI has expiring in 5 months. These batches are pending because they included dead jobs that due to miscommunication with other teams, resulted in the dead jobs being deleted rather than retried. I'm confused by where the 5 months might be coming from.

My goal is to clear out these pending batches that we know will not succeed, so that we can get the admin UI back to being useful for looking at newer batches that may be stuck for different reasons.

[mperham]

30 days is the default. The only way I know to change this value is to monkeypatch the Sidekiq::Batch::EXPIRY constant.

[raybaxter]

Batches with failures show in the UI that they expire in 5 months.

Created at for that batch 2024-05-02 18:19:08 138173/2097152 UTC.

[kwugirl]

Yup, thanks @raybaxter ours looks just like that as well.

[mperham]

This will happen if a batch job dies. Sidekiq Pro extends the batch's expiry to the dead job lifetime so the batch will not expire before any dead batch jobs have been garbage collected. See lib/sidekiq/batch/death.rb line 75.

[stevenharman]

Sidekiq Pro extends the batch's expiry to the dead job lifetime so the batch will not expire before any dead batch jobs have been garbage collected

Ahh, this makes sense! Maybe updating the Wiki will clear this up. 👍

[raybaxter]

My goal is to clear out these pending batches that we know will not succeed, so that we can get the admin UI back to being useful for looking at newer batches that may be stuck for different reasons.

If you have clear criteria for deleting batches, you can delete them from the Rails console:

Sidekiq::BatchSet.new.select {|b| b.description.match /^Full Title/ }.map(&:delete)
Sidekiq::BatchSet.new.select {|b| b.failures > 1 }.map(&:delete)
Sidekiq::BatchSet.new.detect {|b| b.bid == "S9yJ0GD2CW1TGw"}.delete