librelist archives

« back to archive

Separate Sidekiq's

Separate Sidekiq's

From:
Karl Freeman
Date:
2013-04-09 @ 18:07
I'm looking in to adding more background processing into Sidekiq ( as its
working out, real well for other bits of business logic ) however just
wanted to check something.

Would having two separate Sidekiq instances with completely different
workers cause problems if they were sharing one Redis DB? For example, I
send a job which can only be processed by Sidekiq A, what would be the side
effects if Sidekiq B picked that job up? Failure?

I'm assuming different queues might be a way around this but I figured I'd
check if Sidekiq has an awareness of what jobs it can process? and if this
type of splitting of concerns is feasible without meddling with queues (
which should be used for priority )

Cheers
Karl.

Re: [sidekiq] Separate Sidekiq's

From:
Mike Perham
Date:
2013-04-09 @ 18:21
Yeah, use different queues or namespaces.  With different queues you can
manage everything from a single Web UI.  Different namespaces really imply
different applications or environments.


On Tue, Apr 9, 2013 at 11:07 AM, Karl Freeman <karlfreeman@gmail.com> wrote:

> I'm looking in to adding more background processing into Sidekiq ( as its
> working out, real well for other bits of business logic ) however just
> wanted to check something.
>
> Would having two separate Sidekiq instances with completely different
> workers cause problems if they were sharing one Redis DB? For example, I
> send a job which can only be processed by Sidekiq A, what would be the side
> effects if Sidekiq B picked that job up? Failure?
>
> I'm assuming different queues might be a way around this but I figured I'd
> check if Sidekiq has an awareness of what jobs it can process? and if this
> type of splitting of concerns is feasible without meddling with queues (
> which should be used for priority )
>
> Cheers
> Karl.
>

Re: [sidekiq] Separate Sidekiq's

From:
Karl Freeman
Date:
2013-04-11 @ 08:39
Thanks for the responses guys. I should've thought more about this before
sending it.

Different namespaces it is!

Keep up the good work on Sidekiq :)


On 9 April 2013 19:21, Mike Perham <mperham@gmail.com> wrote:

> Yeah, use different queues or namespaces.  With different queues you can
> manage everything from a single Web UI.  Different namespaces really imply
> different applications or environments.
>
>
> On Tue, Apr 9, 2013 at 11:07 AM, Karl Freeman <karlfreeman@gmail.com>wrote:
>
>> I'm looking in to adding more background processing into Sidekiq ( as its
>> working out, real well for other bits of business logic ) however just
>> wanted to check something.
>>
>> Would having two separate Sidekiq instances with completely different
>> workers cause problems if they were sharing one Redis DB? For example, I
>> send a job which can only be processed by Sidekiq A, what would be the side
>> effects if Sidekiq B picked that job up? Failure?
>>
>> I'm assuming different queues might be a way around this but I figured
>> I'd check if Sidekiq has an awareness of what jobs it can process? and if
>> this type of splitting of concerns is feasible without meddling with queues
>> ( which should be used for priority )
>>
>> Cheers
>> Karl.
>>
>
>

Re: [sidekiq] Separate Sidekiq's

From:
Jesse Cooke
Date:
2013-04-09 @ 18:12
Totally feasible and has been done. You just need to name your queues
differently.


On Tue, Apr 9, 2013 at 11:07 AM, Karl Freeman <karlfreeman@gmail.com> wrote:

> I'm looking in to adding more background processing into Sidekiq ( as its
> working out, real well for other bits of business logic ) however just
> wanted to check something.
>
> Would having two separate Sidekiq instances with completely different
> workers cause problems if they were sharing one Redis DB? For example, I
> send a job which can only be processed by Sidekiq A, what would be the side
> effects if Sidekiq B picked that job up? Failure?
>
> I'm assuming different queues might be a way around this but I figured I'd
> check if Sidekiq has an awareness of what jobs it can process? and if this
> type of splitting of concerns is feasible without meddling with queues (
> which should be used for priority )
>
> Cheers
> Karl.
>