librelist archives

Redis timeout/keepalive

Redis timeout/keepalive by John Firebaugh
- Re: [sidekiq] Redis timeout/keepalive by Mike Perham

Redis timeout/keepalive

From:: John Firebaugh
Date:: 2012-09-27 @ 18:45

Is anyone familiar with the details of redis's timeout mechanism and how
that might interact with connection pooling?

I'm seeing writes to redis occasionally time out (Errno::ETIMEDOUT at redis
(3.0.1) lib/redis/connection/ruby.rb:200:in `write'), and I suspect this
may be due to the fact that sidekiq uses long-lived pooled connections and
I have the default `timeout 300` in redis.conf. Am I correct in concluding
that in this configuration, if a connection in the pool goes unused for
five minutes, it's going to be closed by the redis server and then generate
a connection error such as a timeout the next time it's used? And does the
redis gem reconnect automatically or is that broken connection going to
permanently pollute the pool?

Using `timeout 0` in redis.conf would seem to be a solution, but then
apparently one must worry about stale connections piling up on the server,
as mentioned in comment #4 here:
http://code.google.com/p/redis/issues/detail?id=368#c4.

Re: [sidekiq] Redis timeout/keepalive

From:: Mike Perham
Date:: 2012-09-27 @ 19:50

The Redis client handles reconnection internally.  Dalli acts the same
way.  The connection pool purposefully plays dumb here and avoids
doing any health checks / monitoring.

Generally we see Redis timeouts when the system starts swapping.  Some
proactive monitoring here might help.

On Thu, Sep 27, 2012 at 11:45 AM, John Firebaugh
<john.firebaugh@gmail.com> wrote:
> Is anyone familiar with the details of redis's timeout mechanism and how
> that might interact with connection pooling?
>
> I'm seeing writes to redis occasionally time out (Errno::ETIMEDOUT at redis
> (3.0.1) lib/redis/connection/ruby.rb:200:in `write'), and I suspect this may
> be due to the fact that sidekiq uses long-lived pooled connections and I
> have the default `timeout 300` in redis.conf. Am I correct in concluding
> that in this configuration, if a connection in the pool goes unused for five
> minutes, it's going to be closed by the redis server and then generate a
> connection error such as a timeout the next time it's used? And does the
> redis gem reconnect automatically or is that broken connection going to
> permanently pollute the pool?
>
> Using `timeout 0` in redis.conf would seem to be a solution, but then
> apparently one must worry about stale connections piling up on the server,
> as mentioned in comment #4 here:
> http://code.google.com/p/redis/issues/detail?id=368#c4.