Hi all, i am the main developer of the uWSGI project: http://projects.unbit.it/uwsgi/ i have just committed support for zeromq/mongrel2 in the development branch. It is very ugly code mostly because in my company we are investigating mongrel2 usage and we need a test environment ASAP http://projects.unbit.it/uwsgi/browser/proto/zeromq.c To enable mongrel2 support simply add --zeromq option specifying the two sockets url Example (will run the werkzeug testapp): uwsgi --zeromq tcp://127.0.0.1:9999,tcp://127.0.0.1:9998 --module werkzeug.testapp:test_app Only preforking mode is supported, but async mode will be fixed soon (uWSGI does not support edge triggered events so the async api must be extended for zeromq usage). Threading mode will be added in the future. Performance are quite good, (i am using libjansson for JSON parsing), but the code can be optimized a lot. Async file upload is supported. Currently only the python/WSGI plugin can be used as the others must be adapted to the new 0.9.8 api. If you want to test it, be sure to use the 1014 changeset as we are in a big-merge phase where things are expected to broke. To compile it you need the same set of library needed by mongrel2 (libuuid, libzmq), libjansson and the python headers. Every report will be welcomed Thanks -- Roberto De Ioris http://unbit.it
On Tue, Apr 19, 2011 at 11:51:00AM +0200, Roberto De Ioris wrote: > Hi all, i am the main developer of the uWSGI project: > > http://projects.unbit.it/uwsgi/ > > i have just committed support for zeromq/mongrel2 in the development branch. Very cool, and this is with Mongrel2 1.5 right? In trunk we have an initial implementation of a tnetstring version of the transport which would probably make this code much simpler. > Performance are quite good, (i am using libjansson for JSON parsing), but the code can be optimized a lot. > > Async file upload is supported. > > Currently only the python/WSGI plugin can be used as the others must be adapted to the new 0.9.8 api. > > If you want to test it, be sure to use the 1014 changeset as we are in a big-merge phase where things are expected to broke. Neat, I'll try it out. Thanks, and let me know if there's something that you need. -- Zed A. Shaw http://zedshaw.com/
Il giorno 20/apr/2011, alle ore 00.20, Zed A. Shaw ha scritto: > On Tue, Apr 19, 2011 at 11:51:00AM +0200, Roberto De Ioris wrote: >> Hi all, i am the main developer of the uWSGI project: >> >> http://projects.unbit.it/uwsgi/ >> >> i have just committed support for zeromq/mongrel2 in the development branch. > > Very cool, and this is with Mongrel2 1.5 right? In trunk we have an > initial implementation of a tnetstring version of the transport which > would probably make this code much simpler. > Good, i have started implemented it. I have only one question: What is the tnetstring equivalent of the {"METHOD":"JSON", "type":"disconnect"} message ? Thanks -- Roberto De Ioris http://unbit.it
On Wed, Apr 20, 2011 at 11:05:39AM +0200, Roberto De Ioris wrote: > > Il giorno 20/apr/2011, alle ore 00.20, Zed A. Shaw ha scritto: > > Very cool, and this is with Mongrel2 1.5 right? In trunk we have an > > initial implementation of a tnetstring version of the transport which > > would probably make this code much simpler. > Good, i have started implemented it. > > I have only one question: > > What is the tnetstring equivalent of the {"METHOD":"JSON", "type":"disconnect"} message ? There's already a full implementation of tnetstrings in C in the mongrel2 source, and if you build mongrel2 it'll create a file: tests/request_payloads.txt Which has a sequence of json/tnetstring pairs of all the HTTP tests we have for bad or weird requests. You should just be able to reuse the mongrel2 tnetstring code (written by Ryan Kelly) then feed those request_payloads.txt lines in and it should process them all. Also, the cool thing is you can have your uWSGI dynamically handle json or tnetstrings without config. Basically, parse it tnetstrings and since they're backward compatible with netstrings you get either a string or a dict. If it's a string then you've got json so run the json parser. If it's a dict then you're done, it was a tnetstring. I do this in the examples/python/ code. -- Zed A. Shaw http://zedshaw.com/
> On Wed, Apr 20, 2011 at 11:05:39AM +0200, Roberto De Ioris wrote: >> >> Il giorno 20/apr/2011, alle ore 00.20, Zed A. Shaw ha scritto: >> > Very cool, and this is with Mongrel2 1.5 right? In trunk we have an >> > initial implementation of a tnetstring version of the transport which >> > would probably make this code much simpler. >> Good, i have started implemented it. >> >> I have only one question: >> >> What is the tnetstring equivalent of the {"METHOD":"JSON", >> "type":"disconnect"} message ? > > There's already a full implementation of tnetstrings in C in the > mongrel2 source, and if you build mongrel2 it'll create a file: > > tests/request_payloads.txt > > Which has a sequence of json/tnetstring pairs of all the HTTP tests we > have for bad or weird requests. You should just be able to reuse the > mongrel2 tnetstring code (written by Ryan Kelly) then feed those > request_payloads.txt lines in and it should process them all. > > Also, the cool thing is you can have your uWSGI dynamically handle json > or tnetstrings without config. Basically, parse it tnetstrings and > since they're backward compatible with netstrings you get either a > string or a dict. If it's a string then you've got json so run the json > parser. If it's a dict then you're done, it was a tnetstring. > > Just finished all the tests with mongrel2 development release. It survived all of our tests and the best thing is that we can dinamically add/remove uWSGI servers without reconfiguring it. All of our (non-async) apps worked flawlessly and the async upload feature really kick asses. I will now discard all of the current (horrible) json/tnetstring code and move to the Ryan one. Next step is updating the uWSGI-async api to be mongrel2-aware, so we can do something like this: import uwsgi def application(e, s): s('200 Ok', [('Content-Type','text/html')]) uwsgi.set_destinations("10,34,17,34,56") for i in xrange(1,100): uwsgi.wait_fd_read(some_descriptor) yield "Hello<br/>" the set_destinations function should allow you to send the response to multiple clients using zeromq/mongrel2 -- Roberto De Ioris http://unbit.it
On Wed, Apr 20, 2011 at 06:52:30PM +0200, Roberto De Ioris wrote: > Just finished all the tests with mongrel2 development release. > It survived all of our tests and the best thing is that we can dinamically > add/remove uWSGI servers without reconfiguring it. Awesome, and if you fossil up I put a fix for the handler disconnects not being tnetstrings. I, ehem, "tested" it so you might want to confirm that my testing actually works. :-) > import uwsgi > > def application(e, s): > s('200 Ok', [('Content-Type','text/html')]) > > uwsgi.set_destinations("10,34,17,34,56") > > for i in xrange(1,100): > uwsgi.wait_fd_read(some_descriptor) > yield "Hello<br/>" That'd work i believe, and yes you'll run into various things like this since WSGI (and most similar middleware) work on a strict req/rep model instead of the async way Mongrel2 does. -- Zed A. Shaw http://zedshaw.com/
On Wed, Apr 20, 2011 at 11:05, Roberto De Ioris <roberto@unbit.it> wrote: > What is the tnetstring equivalent of the {"METHOD":"JSON", "type":"disconnect"} message ? >>> tnetstring.dumps({'method': 'json', 'type': 'disconnect'}) '37:6:method,4:json,4:type,10:disconnect,}' -- Alexander
Il giorno 20/apr/2011, alle ore 11.17, Alexander Solovyov ha scritto: > On Wed, Apr 20, 2011 at 11:05, Roberto De Ioris <roberto@unbit.it> wrote: >> What is the tnetstring equivalent of the {"METHOD":"JSON", "type":"disconnect"} message ? > >>>> tnetstring.dumps({'method': 'json', 'type': 'disconnect'}) > '37:6:method,4:json,4:type,10:disconnect,}' > > -- > Alexander Yes, that is clear but in the code (function Handler_notify_leave) it looks like that only JSON is used for this feature. -- Roberto De Ioris http://unbit.it
On Wed, Apr 20, 2011 at 11:20:54AM +0200, Roberto De Ioris wrote: > Yes, that is clear but in the code (function Handler_notify_leave) it looks like that only JSON is used for this feature. Ah, yes it is, I'll fix that soon. It should do either based on configuration. It works for me though because the Python lib does what I mentioned in my other email and parses either json or tnetstring on the fly. Thanks for finding that. -- Zed A. Shaw http://zedshaw.com/