librelist archives

« back to archive

Middleware to stop retry on certain errors

Middleware to stop retry on certain errors

From:
James Brennan
Date:
2013-11-05 @ 23:18
Hey All,

I'm trying to write a piece of middleware that will prevent RetryJobs from
putting the job back on the queue if a given error is encountered. For 
example:

class HandleFailure
    def call(worker, msg, queue)
        yield
    rescue ActiveRecord::RecordNotFound
        msg[:retry] = false

        raise e
    end
end

Sidekiq.configure_server do |config|
    config.server_middleware do |chain|
       chain.insert_before Sidekiq::Middleware::Server::RetryJobs, HandleFailure
    end
end

How do I set `msg[:retry] = false` in such a way that it will be passed on
to RetryJobs?
I've look into doing this in the worker itself but ran into the same issue
- msg couldn't be modified after the job has started (or maybe I was doing
something wrong).

Thanks,
James Brennan
Developer
Unbounce.com

Re: [sidekiq] Middleware to stop retry on certain errors

From:
Mike Perham
Date:
2013-11-05 @ 23:29
Generally if there's an error you don't want to retry, your worker should
catch it and return so Sidekiq treats it as a success.

I've considered adding config for "ignore_errors" support a la Airbrake and
Honeybadger but I don't like that it's global.


On Tue, Nov 5, 2013 at 3:18 PM, James Brennan <james@unbounce.com> wrote:

> Hey All,
>
> I'm trying to write a piece of middleware that will prevent RetryJobs from
> putting the job back on the queue if a given error is encountered. For
> example:
>
> class HandleFailure
>     def call(worker, msg, queue)
>         yield
>     rescue ActiveRecord::RecordNotFound
>         msg[:retry] = false
>
>         raise e
>     end
> end
>
> Sidekiq.configure_server do |config|
>     config.server_middleware do |chain|
>        chain.insert_before Sidekiq::Middleware::Server::RetryJobs,
> HandleFailure
>     end
> end
>
> How do I set `msg[:retry] = false` in such a way that it will be passed on
> to RetryJobs?
> I've look into doing this in the worker itself but ran into the same issue
> - msg couldn't be modified after the job has started (or maybe I was doing
> something wrong).
>
> Thanks,
> *James Brennan*
> *Developer*
> Unbounce.com <http://unbounce.com/>
>
>

Re: [sidekiq] Middleware to stop retry on certain errors

From:
James Brennan
Date:
2013-11-05 @ 23:36
Thats exactly what I'm trying accomplish. I need the exception to be 
registered by our application monitoring agent (New Relic), but not have 
the error cause a retry. If the error is rescued by the worker, it won't 
get picked up by the monitoring agent.

If there is no way to have middleware modify the msg, I'll just add the 
functionality to RetryErrors and create a PR if you want the changes.

On 2013-11-05, at 3:29 PM, Mike Perham <mperham@gmail.com> wrote:

> Generally if there's an error you don't want to retry, your worker 
should catch it and return so Sidekiq treats it as a success.
> 
> I've considered adding config for "ignore_errors" support a la Airbrake 
and Honeybadger but I don't like that it's global.
> 
> 
> On Tue, Nov 5, 2013 at 3:18 PM, James Brennan <james@unbounce.com> wrote:
> Hey All,
> 
> I'm trying to write a piece of middleware that will prevent RetryJobs 
from putting the job back on the queue if a given error is encountered. 
For example:
> 
> class HandleFailure
>     def call(worker, msg, queue)
>         yield
>     rescue ActiveRecord::RecordNotFound
>         msg[:retry] = false
> 
>         raise e
>     end
> end
> 
> Sidekiq.configure_server do |config|
>     config.server_middleware do |chain|
>        chain.insert_before Sidekiq::Middleware::Server::RetryJobs, HandleFailure
>     end
> end
> 
> How do I set `msg[:retry] = false` in such a way that it will be passed 
on to RetryJobs?
> I've look into doing this in the worker itself but ran into the same 
issue - msg couldn't be modified after the job has started (or maybe I was
doing something wrong).
> 
> Thanks,
> James Brennan
> Developer
> Unbounce.com
> 
> 

Re: [sidekiq] Middleware to stop retry on certain errors

From:
Paul Walker
Date:
2013-11-06 @ 00:23
why not handle the error in the worker and call airbrake explicitly rather
than trap the retry in middleware just to adhere to airbrake’s specific 
instrumentation details?

On Nov 5, 2013, at 3:36 PM, James Brennan <james@unbounce.com> wrote:

> Thats exactly what I'm trying accomplish. I need the exception to be 
registered by our application monitoring agent (New Relic), but not have 
the error cause a retry. If the error is rescued by the worker, it won't 
get picked up by the monitoring agent.
> 
> If there is no way to have middleware modify the msg, I'll just add the 
functionality to RetryErrors and create a PR if you want the changes.
> 
> On 2013-11-05, at 3:29 PM, Mike Perham <mperham@gmail.com> wrote:
> 
>> Generally if there's an error you don't want to retry, your worker 
should catch it and return so Sidekiq treats it as a success.
>> 
>> I've considered adding config for "ignore_errors" support a la Airbrake
and Honeybadger but I don't like that it's global.
>> 
>> 
>> On Tue, Nov 5, 2013 at 3:18 PM, James Brennan <james@unbounce.com> wrote:
>> Hey All,
>> 
>> I'm trying to write a piece of middleware that will prevent RetryJobs 
from putting the job back on the queue if a given error is encountered. 
For example:
>> 
>> class HandleFailure
>>     def call(worker, msg, queue)
>>         yield
>>     rescue ActiveRecord::RecordNotFound
>>         msg[:retry] = false
>> 
>>         raise e
>>     end
>> end
>> 
>> Sidekiq.configure_server do |config|
>>     config.server_middleware do |chain|
>>        chain.insert_before Sidekiq::Middleware::Server::RetryJobs, 
HandleFailure
>>     end
>> end
>> 
>> How do I set `msg[:retry] = false` in such a way that it will be passed
on to RetryJobs?
>> I've look into doing this in the worker itself but ran into the same 
issue - msg couldn't be modified after the job has started (or maybe I was
doing something wrong).
>> 
>> Thanks,
>> James Brennan
>> Developer
>> Unbounce.com
>> 
>> 
>