Replies: 7 comments 5 replies
-
Use sidekiq_retry_in { 6 * 60 * 60 } and have the job return after early
after the deadline.
…On Sun, Jun 19, 2022 at 11:53 Simon ***@***.***> wrote:
Hi,
TLDR;
What is the best way of having a job retry four times a day until a date?
I am looking for some suggestions or recommendations. I want to create a
job that pulls the status of the payment after each time the bank is
processing payments. BankClearingCycle::Norway.new.next_clearing_cycle
does the work of finding the next next time.
The execution date can be today or it can be a year in the future (imagine
someone registering all the interests for the entire year).
I want it to pull the status until the payment stops being active or if we
are 14 days after the execution date.
I started with pulling for 14 days. It looked like this
class ZtlPollPaymentStatusJob
include Sidekiq::Job
# We want to try for the next 2 weeks after each clearing cycle
sidekiq_options retry: BankClearingCycle::Norway.new.todays_times.size * 14
sidekiq_retry_in do |_count, exception|
next unless exception.is_a? CheckStatusAgainOnNextCycle
Rails.logger.info {
"will retry at next cycle #{BankClearingCycle::Norway.new.next_clearing_cycle} + 5 min"
}
# sidekiq_retry_in should return the int for seconds to wait before executing
# again. First find seconds between next cycle and current time, add a 5
# minutes delay since it takes some time from the cycle to to finish process
# and some jitter
seconds_to_next_cycle = BankClearingCycle::Norway.new.next_clearing_cycle - Time.zone.now
(seconds_to_next_cycle + 5.minutes + rand(5.minutes)).to_i
end
class CheckStatusAgainOnNextCycle < StandardError
end
def perform(id)
basket = ZtlPaymentBasket.find(id)
Ztl::CheckPaymentStatus.call(basket)
basket.reload
Rails.logger.info { "status is #{basket.current_status.process_status}" }
if basket.current_status.process_status == "active"
raise CheckStatusAgainOnNextCycle
end
endend
I think I can change it for:
class ZtlPollPaymentStatusJob
include Sidekiq::Job
# We want to try for the next year after each clearing cycle
sidekiq_options retry: BankClearingCycle::Norway.new.todays_times.size * 365
sidekiq_retry_in do |_count, exception|
next unless exception.is_a? CheckStatusAgainOnNextCycle
Rails.logger.info {
"will retry at next cycle #{BankClearingCycle::Norway.new.next_clearing_cycle} + 5 min"
}
# sidekiq_retry_in should return the int for seconds to wait before executing
# again. First find seconds between next cycle and current time, add a 5
# minutes delay since it takes some time from the cycle to to finish process
# and some jitter
seconds_to_next_cycle = BankClearingCycle::Norway.new.next_clearing_cycle - Time.zone.now
(seconds_to_next_cycle + 5.minutes + rand(5.minutes)).to_i
end
class CheckStatusAgainOnNextCycle < StandardError
end
def perform(id)
basket = ZtlPaymentBasket.find(id)
Ztl::CheckPaymentStatus.call(basket)
basket.reload
Rails.logger.info { "status is #{basket.current_status.process_status}" }
return if basket.current_status.process_status == "active"
if (basket.payments.maximum(:execution_date) + 2.weeks).past?
Rails.logger.info { "payment had execution_date 14 days ago so we stop to retry" }
return
end
raise CheckStatusAgainOnNextCycle
endend
My question: Is this an elegant solution or do you have some other
recommendation?
—
Reply to this email directly, view it on GitHub
<#5397>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAAWX226PNIXMNA6PW3T7LVP3U3NANCNFSM5ZGDY75Q>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Nope. If the job raises an error, Sidekiq will log the backtrace. There are
workarounds as you’ve discovered. You can also replace the default error
handler.
…On Tue, Jun 21, 2022 at 08:45 Simon ***@***.***> wrote:
Is there a way I can avoid logging the backtrace when I raise
CheckStatusAgainOnNextCycle? I see that there are JobRetry::Handled and
JobRetry::Skip, but those seams to do other things.
This works but feels a but dirty
class CheckStatusAgainOnNextCycle < StandardError
def skip_bugsnag
true
end
def backtrace; nil end
end
—
Reply to this email directly, view it on GitHub
<#5397 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAAWX3XIXTM4ZKMCHNE4SDVQFQHXANCNFSM5ZGDY75Q>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Right. I could make improvements for 7.0 but this is a slippery slope. As
soon as you say “don’t log this error” you’ll come up with another edge
case where you do want to log the backtrace. That’s why I have the current
policy on the error handling wiki page.
…On Wed, Jun 22, 2022 at 10:09 Simon ***@***.***> wrote:
Ahh, clever. Like remove Sidekiq::ExceptionHandler::Logger from the
error_handlers and add one of my own.
—
Reply to this email directly, view it on GitHub
<#5397 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAAWXYAPVGPIONHQX2EI6TVQLC2XANCNFSM5ZGDY75Q>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Are there any other ways to retry a job without raiseing an Exception? |
Beta Was this translation helpful? Give feedback.
-
They are internal details due to the two-stage local and global handlers
within job_retry.rb. You can use Skip but it’s not an official public API.
…On Wed, Jun 22, 2022 at 12:43 Simon ***@***.***> wrote:
Are there any other ways to retry a job without raiseing an Exception?
I see that JobRetry::Handled and JobRetry::Skip exists but what are those
for?
—
Reply to this email directly, view it on GitHub
<#5397 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAAWXZKJ44AEHZ5OTIUCWTVQLU6PANCNFSM5ZGDY75Q>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Sorry for going a bit back and fourth. Do you have any opinion on if it's better to retry a job or schedule a new job I talked with some of my colleagues that suggested scheduling a new job instead retry the same jobclass ZtlPollPaymentStatusJob
include Sidekiq::Job
# We want to try for the next year after each clearing cycle
sidekiq_options retry: BankClearingCycle::Norway.new.todays_times.size * 365
sidekiq_retry_in do |_count, exception|
next unless exception.is_a? CheckStatusAgainOnNextCycle
Rails.logger.info {
"will retry at next cycle #{BankClearingCycle::Norway.new.next_clearing_cycle} + 5 min"
}
# sidekiq_retry_in should return the int for seconds to wait before executing
# again. First find seconds between next cycle and current time, add a 5
# minutes delay since it takes some time from the cycle to to finish process
# and some jitter
seconds_to_next_cycle = BankClearingCycle::Norway.new.next_clearing_cycle - Time.zone.now
(seconds_to_next_cycle + 5.minutes + rand(5.minutes)).to_i
end
class CheckStatusAgainOnNextCycle < StandardError
end
def perform(id)
basket = ZtlPaymentBasket.find(id)
Ztl::CheckPaymentStatus.call(basket)
basket.reload
Rails.logger.info { "status is #{basket.current_status.process_status}" }
return if basket.current_status.process_status == "active"
if (basket.payments.maximum(:execution_date) + 2.weeks).past?
Rails.logger.info { "payment had execution_date 14 days ago so we stop to retry" }
return
end
raise CheckStatusAgainOnNextCycle
end
end schedule a new jobclass ZtlPollPaymentStatusJob
include Sidekiq::Job
def perform(id)
basket = ZtlPaymentBasket.find(id)
Ztl::CheckPaymentStatus.call(basket)
basket.reload
Rails.logger.info { "status is #{basket.current_status.process_status}" }
return if basket.current_status.process_status == "active"
if (basket.payments.maximum(:execution_date) + 2.weeks).past?
Rails.logger.info { "payment had execution_date 14 days ago so we stop to retry" }
return
end
ZtlPollPaymentStatusJob
.perform_at(BankClearingCycle::Norway.new.next_clearing_cycle + 5.minutes + rand(5.minutes), id)
end
end |
Beta Was this translation helpful? Give feedback.
-
I think rescheduling the job is possibly a better solution. Exceptions
should be unexpected issues. If something is expected, you should handle it
as normal. If the job runs out of “reschedules”, you could then start
raising errors.
…On Wed, Jun 22, 2022 at 13:11 Simon ***@***.***> wrote:
Sorry for going a bit back and fourth.
Do you have any opinion on if it's better to retry a job or schedule a new
job
with the same arguments?
I talked with some of my colleagues that suggested scheduling a new job
instead
of retrying the same job. Then we would not need to overwrite
sidekiq_retry_in
or error_handler.
retry the same job
class ZtlPollPaymentStatusJob
include Sidekiq::Job
# We want to try for the next year after each clearing cycle
sidekiq_options retry: BankClearingCycle::Norway.new.todays_times.size * 365
sidekiq_retry_in do |_count, exception|
next unless exception.is_a? CheckStatusAgainOnNextCycle
Rails.logger.info {
"will retry at next cycle #{BankClearingCycle::Norway.new.next_clearing_cycle} + 5 min"
}
# sidekiq_retry_in should return the int for seconds to wait before executing
# again. First find seconds between next cycle and current time, add a 5
# minutes delay since it takes some time from the cycle to to finish process
# and some jitter
seconds_to_next_cycle = BankClearingCycle::Norway.new.next_clearing_cycle - Time.zone.now
(seconds_to_next_cycle + 5.minutes + rand(5.minutes)).to_i
end
class CheckStatusAgainOnNextCycle < StandardError
end
def perform(id)
basket = ZtlPaymentBasket.find(id)
Ztl::CheckPaymentStatus.call(basket)
basket.reload
Rails.logger.info { "status is #{basket.current_status.process_status}" }
return if basket.current_status.process_status == "active"
if (basket.payments.maximum(:execution_date) + 2.weeks).past?
Rails.logger.info { "payment had execution_date 14 days ago so we stop to retry" }
return
end
raise CheckStatusAgainOnNextCycle
endend
schedule a new job
class ZtlPollPaymentStatusJob
include Sidekiq::Job
def perform(id)
basket = ZtlPaymentBasket.find(id)
Ztl::CheckPaymentStatus.call(basket)
basket.reload
Rails.logger.info { "status is #{basket.current_status.process_status}" }
return if basket.current_status.process_status == "active"
if (basket.payments.maximum(:execution_date) + 2.weeks).past?
Rails.logger.info { "payment had execution_date 14 days ago so we stop to retry" }
return
end
ZtlPollPaymentStatusJob
.perform_at(BankClearingCycle::Norway.new.next_clearing_cycle, id)
endend
—
Reply to this email directly, view it on GitHub
<#5397 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAAWX6QTHNCXETVEPPK7K3VQLYEVANCNFSM5ZGDY75Q>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Hi,
TLDR;
What is the best way of having a job retry four times a day until a date?
I am looking for some suggestions or recommendations. I want to create a job that pulls the status of the payment after each time the bank is processing payments.
BankClearingCycle::Norway.new.next_clearing_cycle
does the work of finding the next next time.The execution date can be today or it can be a year in the future (imagine someone registering all the interests for the entire year).
I want it to pull the status until the payment stops being active or if we are 14 days after the execution date.
I started with pulling for 14 days. It looked like this
I think I can change it for:
My question: Is this an elegant solution or do you have some other recommendation?
Beta Was this translation helpful? Give feedback.
All reactions