librelist archives

fair queuing

fair queuing by michael j pan
- Re: [mongrel2] fair queuing by Zed A. Shaw

fair queuing

From:: michael j pan
Date:: 2011-08-12 @ 18:10

mongrel2 uses the zmq PUSH socket to forward incoming http requests,
to multiple handlers.  The zmq PUSH socket uses fair queuing, which
would work well in most cases.  but there are specific ones that I
don't think it handles.  Was wondering if I might be mistaken, or if
there are plans to address these cases?

case 1. There are not enough handlers for incoming requests.  The
socket reaches the HWM and blocks.  Does mongrel2 get stuck waiting
for a handler to free up?

case 2.  zmq seems to do round robin load balancing.

http://stackoverflow.com/questions/5268883/how-does-zeromq-determine-order-of-load-balancing
What happens if a handler takes a long time processing a request, and
ends up holding up incoming requests because it has not pulled the
next one?

case 3. If the handler in case 2 is written to be multi-threaded, and
pulls off messages even while it is still processing, it's possible
that a single handler would end up with a long backlog of requests,
while other handlers sit idly.

Mike

Re: [mongrel2] fair queuing

From:: Zed A. Shaw
Date:: 2011-08-13 @ 15:53

On Sat, Aug 13, 2011 at 02:10:51AM +0800, michael j pan wrote:
> case 1. There are not enough handlers for incoming requests.  The
> socket reaches the HWM and blocks.  Does mongrel2 get stuck waiting
> for a handler to free up?

I haven't tested this, but the server should keep going, but that
handler will block until it can send.  It does a poll to see if it can
send it.  But, I should probably test that first.

> case 2.  zmq seems to do round robin load balancing.
> 
http://stackoverflow.com/questions/5268883/how-does-zeromq-determine-order-of-load-balancing
> What happens if a handler takes a long time processing a request, and
> ends up holding up incoming requests because it has not pulled the
> next one?

Sucks to be you. :-)  More seriously, there's been changes to this, but
basically you can fire up more handlers (on other machines) to take on
the new load pretty easily.  It used to be you could just all use the
same identifier, but now you have to either give them NO id or give them
all different consistent ids.  If you do that, then when you get
overloaded you should be able to spin up/down handlers as you need.

Otherwise, yeah in *all* systems if your handlers are overloaded they'll
block requests.

Next, the only thing that happens otherwise, is there's timeout values
you can set in the mongrel2.conf which will kill clients that aren't
doing anything.  It's set really conservatively at 120 seconds, but you
could drop it down and then if things are taking too long clients will
get closed off.

> case 3. If the handler in case 2 is written to be multi-threaded, and
> pulls off messages even while it is still processing, it's possible
> that a single handler would end up with a long backlog of requests,
> while other handlers sit idly.

Yep, that's kind of why I don't like using threads and instead use
processes.  The failure case for a lot of threads handling 0mq messages
is that, if there's a bug and one process goes down, you can lose a
whole lot of requests.  The failure case for a lot of processes is, if
there's a bug, then you only lose one or two requests when any process
dies.  In theory that should be safer.

-- 
Zed A. Shaw
http://zedshaw.com/