Re: [sidekiq] High Latency for calls to perform_async on RedisToGo instance
- Jack Royal-Gordon
- 2015-10-20 @ 18:22
I should add that other Sidekiq calls, such as
Sidekiq::RetrySet.new.count, complete acceptably fast (1-2 seconds), so
this problem seems to be restricted to adding jobs to a queue.
> On Oct 20, 2015, at 11:21 AM, Jack Royal-Gordon <firstname.lastname@example.org> wrote:
> A couple days ago I started experiencing long run times in certain jobs.
It turns out that suddenly perform_async calls are taking 10 seconds to
complete. Since I’m queueing 75K jobs, you can imagine the long run time.
I’m running on Heroku with my Redis database on RedisToGo. I’ve contacted
them, they’ve looked and see nothing in particular out of whack with my
instance (which I have tried restarting), but they asked the following
question which I don’t feel I can answer. Can one of you answer this
> Also, I’d appreciate If someone could provide specifics as to which
Redis calls are used when queueing a job in a specific queue (not the
Scheduled queue), so I can pass that info along to RedisToGo Support.
> However, I did notice SMEMBERS calls with significant time spent per
call as we are fetching larger content per key.
> Can you inform me why a SCAN could not be used (HSCAN, SSCAN, ZSCAN) to
reduce overall latency with these fetches? I will continue to review and
see if we can tune anything else in this instance.