librelist archives

« back to archive

Adding reliable_fetch causes Sidekiq to not pick up jobs

Adding reliable_fetch causes Sidekiq to not pick up jobs

From:
Jonathan Hyman
Date:
2013-11-08 @ 21:28
Hey all,

I’m trying to set up reliable fetch but for some reason, reliable fetch
causes the worker to not grab any work.

I have a Sidekiq.configure_server block where if I comment out the line

require 'sidekiq/pro/reliable_fetch'

then it works fine. Here’s an excerpt from the logs.

# With reliable_fetch required

> RAILS_ENV=production bundle exec sidekiq -e production -i 7 -q data_process &

 9:12:43.418336709] (21933) [INFO] Booting Sidekiq 2.16.1 with redis
options {:url=>"REDACTED", :namespace=>"REDACTED", :pool_timeout=>10,
:network_timeout=>10}
[ 9:12:43.420389939] (21933) [INFO] Running in ruby 2.0.0p247
(2013-06-27 revision 41674) [x86_64-linux]
[ 9:12:43.420469714] (21933) [INFO] Sidekiq Pro 1.2.5, commercially
licensed.  Thanks for your support!
[ 9:12:43.420532688] (21933) [INFO] Starting processing, hit Ctrl-C to stop
[ 9:12:43.425150806] (21933) [DEBUG] {:queues=>["data_process"],
:concurrency=>25, :require=>".", :environment=>"production",
:timeout=>8, :profile=>false, :index=>7, :strict=>true,
 :notifications=>{}, :backup_limit=>1000,
:fetch=>Sidekiq::Pro::ReliableFetch, :tag=>"REDACTED"}
[ 9:12:43.629315857] (21933) [INFO] ReliableFetch activated

# No jobs are picked up. Meanwhile, our queue grows.

[ 9:13:30.758581777] (21933) [DEBUG] Got INT signal
[ 9:13:30.758893259] (21933) [INFO] Shutting down

If I comment out that line:

> RAILS_ENV=production bundle exec sidekiq -e production -i 7 -q data_process &

 9:17:23.540943704] (22757) [INFO] Booting Sidekiq 2.16.1 with redis
options {:url=>"REDACTED", :namespace=>"REDACTED", :pool_timeout=>10,
:network_timeout=>10}
[ 9:17:23.545718712] (22757) [INFO] Running in ruby 2.0.0p247
(2013-06-27 revision 41674) [x86_64-linux]
[ 9:17:23.545803885] (22757) [INFO] Sidekiq Pro 1.2.5, commercially
licensed.  Thanks for your support!
[ 9:17:23.545869223] (22757) [INFO] Starting processing, hit Ctrl-C to stop
[ 9:17:23.784453445] (22757) [DEBUG] {:queues=>["data_process"],
:concurrency=>25, :require=>".", :environment=>"production",
:timeout=>8, :profile=>false, :index=>7, :strict=>true,
 :notifications=>{}, :backup_limit=>1000, :tag=>"REDACTED"}

 # It picks up a job immediately
[ 9:17:24.757436345] (22757) [INFO] start

Any ideas?

Thanks,
Jon

Re: Adding reliable_fetch causes Sidekiq to not pick up jobs

From:
Jonathan Hyman
Date:
2013-11-08 @ 22:50
A quick update: It seems like reliable fetch has a bug in it, at least in
my version of Ruby. I modified the code to grab jobs back, but they appear
to not get processed. Mike, I'll follow up with you with a separate email
about the bug.

On Fri, Nov 8, 2013 at 4:28 PM, Jonathan Hyman <hyman.jon@gmail.com> wrote:

> Hey all,
>
> I’m trying to set up reliable fetch but for some reason, reliable fetch
> causes the worker to not grab any work.
>
> I have a Sidekiq.configure_server block where if I comment out the line
>
> require 'sidekiq/pro/reliable_fetch'
>
> then it works fine. Here’s an excerpt from the logs.
>
> # With reliable_fetch required
>
> > RAILS_ENV=production bundle exec sidekiq -e production -i 7 -q data_process &
>
>  9:12:43.418336709] (21933) [INFO] Booting Sidekiq 2.16.1 with redis 
options {:url=>"REDACTED", :namespace=>"REDACTED", :pool_timeout=>10, 
:network_timeout=>10}
> [ 9:12:43.420389939] (21933) [INFO] Running in ruby 2.0.0p247 
(2013-06-27 revision 41674) [x86_64-linux]
> [ 9:12:43.420469714] (21933) [INFO] Sidekiq Pro 1.2.5, commercially 
licensed.  Thanks for your support!
> [ 9:12:43.420532688] (21933) [INFO] Starting processing, hit Ctrl-C to stop
> [ 9:12:43.425150806] (21933) [DEBUG] {:queues=>["data_process"], 
:concurrency=>25, :require=>".", :environment=>"production", :timeout=>8, 
:profile=>false, :index=>7, :strict=>true,
>  :notifications=>{}, :backup_limit=>1000, 
:fetch=>Sidekiq::Pro::ReliableFetch, :tag=>"REDACTED"}
> [ 9:12:43.629315857] (21933) [INFO] ReliableFetch activated
>
> # No jobs are picked up. Meanwhile, our queue grows.
>
> [ 9:13:30.758581777] (21933) [DEBUG] Got INT signal
> [ 9:13:30.758893259] (21933) [INFO] Shutting down
>
> If I comment out that line:
>
> > RAILS_ENV=production bundle exec sidekiq -e production -i 7 -q data_process &
>
>  9:17:23.540943704] (22757) [INFO] Booting Sidekiq 2.16.1 with redis 
options {:url=>"REDACTED", :namespace=>"REDACTED", :pool_timeout=>10, 
:network_timeout=>10}
> [ 9:17:23.545718712] (22757) [INFO] Running in ruby 2.0.0p247 
(2013-06-27 revision 41674) [x86_64-linux]
> [ 9:17:23.545803885] (22757) [INFO] Sidekiq Pro 1.2.5, commercially 
licensed.  Thanks for your support!
> [ 9:17:23.545869223] (22757) [INFO] Starting processing, hit Ctrl-C to stop
> [ 9:17:23.784453445] (22757) [DEBUG] {:queues=>["data_process"], 
:concurrency=>25, :require=>".", :environment=>"production", :timeout=>8, 
:profile=>false, :index=>7, :strict=>true,
>  :notifications=>{}, :backup_limit=>1000, :tag=>"REDACTED"}
>
>  # It picks up a job immediately
> [ 9:17:24.757436345] (22757) [INFO] start
>
> Any ideas?
>
> Thanks,
> Jon
>



On Fri, Nov 8, 2013 at 4:28 PM, Jonathan Hyman <hyman.jon@gmail.com> wrote:

> Hey all,
>
> I’m trying to set up reliable fetch but for some reason, reliable fetch
> causes the worker to not grab any work.
>
> I have a Sidekiq.configure_server block where if I comment out the line
>
> require 'sidekiq/pro/reliable_fetch'
>
> then it works fine. Here’s an excerpt from the logs.
>
> # With reliable_fetch required
>
> > RAILS_ENV=production bundle exec sidekiq -e production -i 7 -q data_process &
>
>  9:12:43.418336709] (21933) [INFO] Booting Sidekiq 2.16.1 with redis 
options {:url=>"REDACTED", :namespace=>"REDACTED", :pool_timeout=>10, 
:network_timeout=>10}
> [ 9:12:43.420389939] (21933) [INFO] Running in ruby 2.0.0p247 
(2013-06-27 revision 41674) [x86_64-linux]
> [ 9:12:43.420469714] (21933) [INFO] Sidekiq Pro 1.2.5, commercially 
licensed.  Thanks for your support!
> [ 9:12:43.420532688] (21933) [INFO] Starting processing, hit Ctrl-C to stop
> [ 9:12:43.425150806] (21933) [DEBUG] {:queues=>["data_process"], 
:concurrency=>25, :require=>".", :environment=>"production", :timeout=>8, 
:profile=>false, :index=>7, :strict=>true,
>  :notifications=>{}, :backup_limit=>1000, 
:fetch=>Sidekiq::Pro::ReliableFetch, :tag=>"REDACTED"}
> [ 9:12:43.629315857] (21933) [INFO] ReliableFetch activated
>
> # No jobs are picked up. Meanwhile, our queue grows.
>
> [ 9:13:30.758581777] (21933) [DEBUG] Got INT signal
> [ 9:13:30.758893259] (21933) [INFO] Shutting down
>
> If I comment out that line:
>
> > RAILS_ENV=production bundle exec sidekiq -e production -i 7 -q data_process &
>
>  9:17:23.540943704] (22757) [INFO] Booting Sidekiq 2.16.1 with redis 
options {:url=>"REDACTED", :namespace=>"REDACTED", :pool_timeout=>10, 
:network_timeout=>10}
> [ 9:17:23.545718712] (22757) [INFO] Running in ruby 2.0.0p247 
(2013-06-27 revision 41674) [x86_64-linux]
> [ 9:17:23.545803885] (22757) [INFO] Sidekiq Pro 1.2.5, commercially 
licensed.  Thanks for your support!
> [ 9:17:23.545869223] (22757) [INFO] Starting processing, hit Ctrl-C to stop
> [ 9:17:23.784453445] (22757) [DEBUG] {:queues=>["data_process"], 
:concurrency=>25, :require=>".", :environment=>"production", :timeout=>8, 
:profile=>false, :index=>7, :strict=>true,
>  :notifications=>{}, :backup_limit=>1000, :tag=>"REDACTED"}
>
>  # It picks up a job immediately
> [ 9:17:24.757436345] (22757) [INFO] start
>
> Any ideas?
>
> Thanks,
> Jon
>