librelist archives

The case of the missing header "PATTERN"

From:

Alfred Tascon

Date:

2011-09-28 @ 02:31

Im finding the header "PATTERN" is not ending up in the headers of the
request object.

I found it goes missing if the same TCP connection is used for both a proxy
and handler paths.

It can be reproduced by running these following two apps for backends from
the examples directory:
1 - http0mq/http.py
        nohup python -n http0mq/http.py 2>&1 > /dev/null &
2 - chat/www.py - Obviously you need the python 'web.py' egg to be
installed.
        nohup python -n chat/www.py 2>&1 >/dev/null &

Using the configuration file in the examples directory
"configs/mongrel2.conf" fire up Mongrel2:
m2sh load -db config.sqlite -config configs/mongrel2.conf
m2sh start -db config.sqlite -name main

And then in a browser (i used firefox 6) type the following:
http://127.0.0.1:6767/handlertest
Here you will notice that the header 'PATTERN' is there.

then type:
http://127.0.0.1:6767/
You get the random number of "Hello World"s.

Now type (hopefully not too much time has passed so the browser is using the
same TCP connection):
http://127.0.0.1:6767/handlertest
Now you will see that the header 'PATTERN' is missing.

If you were to wait for the TCP connection to reset between the last two
calls the header "PATTERN" will be there on the last call.

Chrome gives similar results.

The header "PATTERN" is very useful as I use it to trim the part of the path
that was used to "route" to the application.
Since the rest of the path is variable makes my parser simpler not needing
to know how the final configuration, with the other apps, etc.., was setup.


Alfred

PS
Im using Mongrel version 1.7.5 and its python layer under python 2.7.

2011/9/27 Alfred Tascon <atascon@gmail.com>:
> Im finding the header "PATTERN" is not ending up in the headers of the
> request object.
> I found it goes missing if the same TCP connection is used for both a proxy
> and handler paths.
> It can be reproduced by running these following two apps for backends from
> the examples directory:
> 1 - http0mq/http.py
>         nohup python -n http0mq/http.py 2>&1 > /dev/null &
> 2 - chat/www.py - Obviously you need the python 'web.py' egg to be
> installed.
>         nohup python -n chat/www.py 2>&1 >/dev/null &
> Using the configuration file in the examples directory
> "configs/mongrel2.conf" fire up Mongrel2:
> m2sh load -db config.sqlite -config configs/mongrel2.conf
> m2sh start -db config.sqlite -name main
> And then in a browser (i used firefox 6) type the following:
> http://127.0.0.1:6767/handlertest
> Here you will notice that the header 'PATTERN' is there.
> then type:
> http://127.0.0.1:6767/
> You get the random number of "Hello World"s.
> Now type (hopefully not too much time has passed so the browser is using the
> same TCP connection):
> http://127.0.0.1:6767/handlertest
> Now you will see that the header 'PATTERN' is missing.
> If you were to wait for the TCP connection to reset between the last two
> calls the header "PATTERN" will be there on the last call.
> Chrome gives similar results.
> The header "PATTERN" is very useful as I use it to trim the part of the path
> that was used to "route" to the application.
> Since the rest of the path is variable makes my parser simpler not needing
> to know how the final configuration, with the other apps, etc.., was setup.
>

I can confirm this, and here is what I saw when testing. I used the
develop branch to do the tests.

I added one line to log 3 infos: conn_id, backend_type and the value
of pattern. This problem only happens
when the second backend (www.py, which is a proxy) is running.

Here is the log line I added:

 log_info("conn-id=%d, backend=%d, Pattern= %s\n",
Register_id_for_fd(IOBuf_fd(conn->iob)),
                                          conn->req->action->type,
                                          bdata(conn->req->pattern));

It was added inside connection_route_request (connection.c:124). When
I make a request to the first URL this is what I see on the
terminal where mongrel2 is running:

[INFO] (src/connection.c:125) conn-id=1, backend=1, Pattern= /handlertest
[INFO] (src/connection.c:125) conn-id=1, backend=2, Pattern= /

As far as I understand, mongrel2 sent the message to *both* backends,
the handler and the proxy. And then I make the request to the the
second URL, and when I come back to the first and reload, this is what
mongrel2 prints:

[INFO] (src/connection.c:125) conn-id=1, backend=2, Pattern= /

It does not print one line for the handler backend (the one I just
reload on the browser), however it seems that the handler does
receives the message, because the browser updates the page. I tried
with different paths: /handlertest/{a,b,c,...}, same behavior.

So I changed the proxy path from "/" to "/p" and the problem does not
happen. Then I renamed the "/mp3stream" path to "/", mongrel2 again
sends the same message to both backends but the PATTERN header is
always present on both message. Another detail, with this last
configuration, when I request "/handlertest" mongrel prints 2 lines on
the terminal, one for each backend:

[INFO] (src/connection.c:125) conn-id=2, backend=1, Pattern= /handlertest
[INFO] (src/connection.c:125) conn-id=2, backend=1, Pattern= /

I monitored both terminals, and confirmed that both printed new
information. That's how I confirmed the message was sent to both
backends, and not two different messages sent to the same backend
(with different PATTERNs).

So it seems to be an issue with handler and proxy routes when the
route values have a common prefix, maybe?

One last test I also did: Changes the proxy route from "/" to "/han"
and re-made the test. All OK! Absolutely no problems. Maybe some
specific issue with the root route being proxy backed?

So, very long mail (I apologize), so many different tests, but I hope
it helps to solve to this problem.

I have yet to study mongrel2's source code much more, but since I
found different information than Alfred I though it would be valuable
to post here. If I have some time tomorrow I will try to take a look
at this a little more.

Thanks.


-- 
Dalton Barreto
http://daltonmatos.com

2011/9/27 Alfred Tascon <atascon@gmail.com>:
> Im finding the header "PATTERN" is not ending up in the headers of the
> request object.
> I found it goes missing if the same TCP connection is used for both a proxy
> and handler paths.
> It can be reproduced by running these following two apps for backends from
> the examples directory:
> 1 - http0mq/http.py
>         nohup python -n http0mq/http.py 2>&1 > /dev/null &
> 2 - chat/www.py - Obviously you need the python 'web.py' egg to be
> installed.
>         nohup python -n chat/www.py 2>&1 >/dev/null &
> Using the configuration file in the examples directory
> "configs/mongrel2.conf" fire up Mongrel2:
> m2sh load -db config.sqlite -config configs/mongrel2.conf
> m2sh start -db config.sqlite -name main
> And then in a browser (i used firefox 6) type the following:
> http://127.0.0.1:6767/handlertest
> Here you will notice that the header 'PATTERN' is there.
> then type:
> http://127.0.0.1:6767/
> You get the random number of "Hello World"s.
> Now type (hopefully not too much time has passed so the browser is using the
> same TCP connection):
> http://127.0.0.1:6767/handlertest
> Now you will see that the header 'PATTERN' is missing.
> If you were to wait for the TCP connection to reset between the last two
> calls the header "PATTERN" will be there on the last call.
> Chrome gives similar results.
> The header "PATTERN" is very useful as I use it to trim the part of the path
> that was used to "route" to the application.
> Since the rest of the path is variable makes my parser simpler not needing
> to know how the final configuration, with the other apps, etc.., was setup.
>
> Alfred
> PS
> Im using Mongrel version 1.7.5 and its python layer under python 2.7.

Hello Alfred,

I also have a python handler, wsgid [1]. I will try to take a look at
this during the weekend
and report back.

thanks.


[1] http://wsgid.com

-- 
Dalton Barreto
http://daltonmatos.com