librelist archives

« back to archive

Reloading Tir Handlers

Reloading Tir Handlers

From:
Matt Towers
Date:
2011-02-01 @ 22:19
I think I saw this mentioned before as a known issue somewhere but I can't
seem to locate it now...

After reloading Tir handlers (CTRL-C), the first request routed to a given
handler hangs.  Subsequent requests behave as expected.

Anyone have any ideas on a workaround, or is the problem between my 
keyboard and chair?


✈ Matt


Re: [mongrel2] Reloading Tir Handlers

From:
Zed A. Shaw
Date:
2011-02-02 @ 08:32
On Tue, Feb 01, 2011 at 02:19:04PM -0800, Matt Towers wrote:
> I think I saw this mentioned before as a known issue somewhere but I 
can't seem to locate it now...

I have to delve into this because I never hit this.  I need to come up
with more reproducible test cases than just "curl a ton and hit ctrl-c".

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] Reloading Tir Handlers

From:
Matt Towers
Date:
2011-02-02 @ 17:49
Here's a stack trace that is dumped on the first request after a CTRL-C


REQUEST /zip:0	Wed Feb  2 17:47:18 2011	/zip	GET	
APP-7d57f04267cfbe8cd55c7b7b8279449d
^C
lua: /usr/local/share/lua/5.1//mongrel2/connection.lua:65: interrupted!
stack traceback:
	[C]: in function 'recv'
	/usr/local/share/lua/5.1//mongrel2/connection.lua:65: in function 'recv'
	/usr/local/share/lua/5.1//tir/engine.lua:103: in function 'run'
	/usr/local/share/lua/5.1//tir/engine.lua:170: in function 'start'
	/usr/local/share/lua/5.1//tir/engine.lua:178: in function 'stateless'
	app/zipline.lua:13: in main chunk
	[C]: ?
CHILD DIED app/zipline.lua PID 19854:	exited
Started app/zipline.lua PID 19858
CONNECTING	/zip	38f857b8-cbaa-4b58-9271-0d36c27813c4	tcp://127.0.0.1:9990	
tcp://127.0.0.1:9989


✈ Matt



On Feb 2, 2011, at 00:32 , Zed A. Shaw wrote:

> On Tue, Feb 01, 2011 at 02:19:04PM -0800, Matt Towers wrote:
>> I think I saw this mentioned before as a known issue somewhere but I 
can't seem to locate it now...
> 
> I have to delve into this because I never hit this.  I need to come up
> with more reproducible test cases than just "curl a ton and hit ctrl-c".
> 
> -- 
> Zed A. Shaw
> http://zedshaw.com/

Re: [mongrel2] Reloading Tir Handlers

From:
Zed A. Shaw
Date:
2011-02-02 @ 17:58
On Wed, Feb 02, 2011 at 09:49:55AM -0800, Matt Towers wrote:
> Here's a stack trace that is dumped on the first request after a CTRL-C
> 
> 
> REQUEST /zip:0	Wed Feb  2 17:47:18 2011	/zip	GET	
APP-7d57f04267cfbe8cd55c7b7b8279449d

Ok, and does this happen all the time, or just sometimes?  What version
of 0mq are you using?

I'm wondering if I can use some lua_signal to catch the ^C and then exit
after the recv rather than just leaving it to lua.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] Reloading Tir Handlers

From:
Matt Towers
Date:
2011-02-02 @ 18:33
I can reproduce it consistently.

1.  curl http://localhost:6767/zip
2.  CTRL-C
3.  curl http://localhost:6767/zip



My installed 0mq version:  2.0.10

✈ Matt



On Feb 2, 2011, at 09:58 , Zed A. Shaw wrote:

> On Wed, Feb 02, 2011 at 09:49:55AM -0800, Matt Towers wrote:
>> Here's a stack trace that is dumped on the first request after a CTRL-C
>> 
>> 
>> REQUEST /zip:0	Wed Feb  2 17:47:18 2011	/zip	GET	
APP-7d57f04267cfbe8cd55c7b7b8279449d
> 
> Ok, and does this happen all the time, or just sometimes?  What version
> of 0mq are you using?
> 
> I'm wondering if I can use some lua_signal to catch the ^C and then exit
> after the recv rather than just leaving it to lua.
> 
> -- 
> Zed A. Shaw
> http://zedshaw.com/

Re: [mongrel2] Reloading Tir Handlers

From:
joshua simmons
Date:
2011-02-02 @ 19:03
This isn't the thing with zmq_threads > 1 is it?
On Feb 3, 2011 5:34 AM, "Matt Towers" <matt@ziplinegames.com> wrote:
> I can reproduce it consistently.
>
> 1. curl http://localhost:6767/zip
> 2. CTRL-C
> 3. curl http://localhost:6767/zip
>
>
>
> My installed 0mq version: 2.0.10
>
> ✈ Matt
>
>
>
> On Feb 2, 2011, at 09:58 , Zed A. Shaw wrote:
>
>> On Wed, Feb 02, 2011 at 09:49:55AM -0800, Matt Towers wrote:
>>> Here's a stack trace that is dumped on the first request after a CTRL-C
>>>
>>>
>>> REQUEST /zip:0 Wed Feb 2 17:47:18 2011 /zip GET
APP-7d57f04267cfbe8cd55c7b7b8279449d
>>
>> Ok, and does this happen all the time, or just sometimes? What version
>> of 0mq are you using?
>>
>> I'm wondering if I can use some lua_signal to catch the ^C and then exit
>> after the recv rather than just leaving it to lua.
>>
>> --
>> Zed A. Shaw
>> http://zedshaw.com/
>

Re: [mongrel2] Reloading Tir Handlers

From:
Zed A. Shaw
Date:
2011-02-02 @ 22:39
On Thu, Feb 03, 2011 at 06:03:38AM +1100, joshua simmons wrote:
> This isn't the thing with zmq_threads > 1 is it?

No I don't think that's related.  That was a crash, but yeah maybe the
Lua side needs threads=1 as well.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] Reloading Tir Handlers

From:
James Dennis
Date:
2011-02-01 @ 22:32
The zmq docs mention cases where messages are lost due to socket setup
times. Specifically, some of the first messages in a stream.

Perhaps related?



On Feb 1, 2011, at 5:19 PM, Matt Towers <matt@ziplinegames.com> wrote:

I think I saw this mentioned before as a known issue somewhere but I can't
seem to locate it now...

After reloading Tir handlers (CTRL-C), the first request routed to a given
handler hangs.  Subsequent requests behave as expected.

Anyone have any ideas on a workaround, or is the problem between my keyboard
and chair?


✈ Matt

Re: [mongrel2] Reloading Tir Handlers

From:
Henry Finucane
Date:
2011-02-01 @ 22:33
On Tue, Feb 1, 2011 at 2:19 PM, Matt Towers <matt@ziplinegames.com> wrote:
> I think I saw this mentioned before as a known issue somewhere but I can't
> seem to locate it now...

There was a thread about lost requests with the Python handler last
week, maybe that's what you're remembering?

> After reloading Tir handlers (CTRL-C), the first request routed to a given
> handler hangs.  Subsequent requests behave as expected.
> Anyone have any ideas on a workaround, or is the problem between my keyboard
> and chair?
>
> ✈ Matt
>
>
>
-- 
hf

Re: [mongrel2] Reloading Tir Handlers

From:
Ryan Kelly
Date:
2011-02-01 @ 23:15
On Tue, 2011-02-01 at 14:33 -0800, Henry Finucane wrote:
> On Tue, Feb 1, 2011 at 2:19 PM, Matt Towers <matt@ziplinegames.com> wrote:
> > I think I saw this mentioned before as a known issue somewhere but I can't
> > seem to locate it now...
> 
> There was a thread about lost requests with the Python handler last
> week, maybe that's what you're remembering?

The conclusion from that thread was to make sure you set the socket
ident, to take advantage of zmq's buffering capabilities.

> > After reloading Tir handlers (CTRL-C), the first request routed to a given
> > handler hangs.  Subsequent requests behave as expected.
> > Anyone have any ideas on a workaround, or is the problem between my keyboard
> > and chair?

I see the same thing with my m2wsgi handler if I don't set a socket
ident.  If I kill the handler and reload it, the next request hangs but
subsequent requests go through OK.

Setting the socket identity on both send and recv sockets in the handler
makes this go away.  Not sure how to do that in Tir but it sounds like
what you need.

(I still run into other, more subtle situations where setting the socket
ident doesn't save you from lost requests, but I suspect this is a
rather fundamental limitation of zmq.  Setting the ident certainly helps
a *lot*)


   Ryan

-- 
Ryan Kelly
http://www.rfk.id.au  |  This message is digitally signed. Please visit
ryan@rfk.id.au        |  http://www.rfk.id.au/ramblings/gpg/ for details