Alright everyone, it's time to kill some sacred cows. I've been collecting feedback and using Mongrel2 myself on a project since the 1.0 release. I've got a few things that suck (or sucked) in 1.0 and I'd like to collect up more. Anything you've got reply to this and say your piece. Let's put a bit of structure around this and say keep the debate down and just talk about what sucks and what they want right now. Let everyone get their opinions out, then I'll collect them up and work out features and we can talk about those. Here's my list: * m2sh in Python doesn't work. First people who are on broke ass operating system slike Solaris or RHEL are stuck in Python 2.4 or worse. Second everyone does their package installs totally different so dependencies suck. Third there's some language users who just hate Python irrationally. * We need a more fine grain error.log with warns and timestamps. Trying to find out when something happened is too difficult. * HTTP/1.0 and Connection:close still don't work. * I fixed this one, but the damn manual still had people trying to download with fossil when they could have just grabbed a tar.bz2. * Having to specify the stupid database and config file to m2sh every time is lame. Use a reasonable default of config.sqlite and config.py. * Mongrel2 is *already* a reverse proxy, so why the hell not just let people use it to go grab stuff on the internet from their handlers? Put some of that awesome kegogi into Mongrel2. * People are starting to want to do things with it like session affinity and rate limiting. We need filters very soon. * Some people still can't handle manually installing dependencies, so we probably should include a git checkout of zeromq or some kind of setup script or something so they don't balk at having to download other software first. * Each supported language has absolutely no deployment or install story. None. No way people could be expected to use them right if we haven't used them and haven't told them how best to do it. * Python m2py needs WSGI bad. Real bad. * Mongrel2 needs WebSockets. About every 3rd person asks for this. * There needs to be some of the demos of doing different ZeroMQ architecturees in our docs and examples. * Switch to making releases by default, rather than full debug installs. * Bite the bullet and mirror the fossil repo under git and hg. Core contributors still use fossil but lusers love their git and actually refuse to use software that isn't hosted with git. * More kegogi and client side testing. Found a bug because tnftp doesn't do things right so I need to have scripts galore that test this out. * More unix platforms. I can do netbsd, osx, most linux, others? * More control port abilities, and hell why not have m2sh try to do everything through the control port? * A better workflow for web devs. Right now they gotta work like an ops person which sucks when you're stuck in an a text editor and restarting mongrel2 to update your config. Get it to where you edit a file and "magic" happens and the config is uploaded. * Break the big ass manual up into a small number of files by section so it loads faster and is easier to browse. * Expand the quickstart so it's basically the tl;dr version of the manual. * Performance tune this thing to death. Alright that's my list. What's yours? Remember, anything you think sucks or want, and nobody can disagree with you (yet). So let it all out. -- Zed A. Shaw http://zedshaw.com/
What do you think about bringing some Lua into Mongrel2? Perhaps m2sh could be rewritten in Lua. I'm not saying it is easy, but it might ease some of the Python issues. There are sqlite3 bindings at http://luaforge.net/projects/luasqlite/. Also - and this may just be plain crazy - what do you think about allowing users to write custom filters or plugins in Lua? Someone asked about sticky sessions, for example. Instead of writing this filter in C, could a user write it in Lua? Supposedly Lua is fast. I don't know if it is fast enough for a Mongrel2 filter, though. But Lua is a language that is very accessible to web developers. For simple tasks, it is almost identical to javascript. - Devon On Sep 5, 2010, at 2:22 PM, Zed A. Shaw wrote: > Alright everyone, it's time to kill some sacred cows. I've been > collecting feedback and using Mongrel2 myself on a project since the 1.0 > release. I've got a few things that suck (or sucked) in 1.0 and I'd > like to collect up more. Anything you've got reply to this and say your > piece. > > Let's put a bit of structure around this and say keep the debate down > and just talk about what sucks and what they want right now. Let > everyone get their opinions out, then I'll collect them up and work out > features and we can talk about those. > > Here's my list: > > * m2sh in Python doesn't work. First people who are on broke ass > operating system slike Solaris or RHEL are stuck in Python 2.4 or worse. > Second everyone does their package installs totally different so > dependencies suck. Third there's some language users who just hate > Python irrationally. > * We need a more fine grain error.log with warns and timestamps. Trying > to find out when something happened is too difficult. > * HTTP/1.0 and Connection:close still don't work. > * I fixed this one, but the damn manual still had people trying to > download with fossil when they could have just grabbed a tar.bz2. > * Having to specify the stupid database and config file to m2sh every > time is lame. Use a reasonable default of config.sqlite and > config.py. > * Mongrel2 is *already* a reverse proxy, so why the hell not just let > people use it to go grab stuff on the internet from their handlers? Put > some of that awesome kegogi into Mongrel2. > * People are starting to want to do things with it like session > affinity and rate limiting. We need filters very soon. > * Some people still can't handle manually installing dependencies, so we > probably should include a git checkout of zeromq or some kind of setup > script or something so they don't balk at having to download other > software first. > * Each supported language has absolutely no deployment or install story. > None. No way people could be expected to use them right if we haven't > used them and haven't told them how best to do it. > * Python m2py needs WSGI bad. Real bad. > * Mongrel2 needs WebSockets. About every 3rd person asks for this. > * There needs to be some of the demos of doing different ZeroMQ > architecturees in our docs and examples. > * Switch to making releases by default, rather than full debug installs. > * Bite the bullet and mirror the fossil repo under git and hg. Core > contributors still use fossil but lusers love their git and actually > refuse to use software that isn't hosted with git. > * More kegogi and client side testing. Found a bug because tnftp > doesn't do things right so I need to have scripts galore that test > this out. > * More unix platforms. I can do netbsd, osx, most linux, others? > * More control port abilities, and hell why not have m2sh try to do > everything through the control port? > * A better workflow for web devs. Right now they gotta work like an ops > person which sucks when you're stuck in an a text editor and restarting > mongrel2 to update your config. Get it to where you edit a file and > "magic" happens and the config is uploaded. > * Break the big ass manual up into a small number of files by section so > it loads faster and is easier to browse. > * Expand the quickstart so it's basically the tl;dr version of the > manual. > * Performance tune this thing to death. > > > Alright that's my list. What's yours? Remember, anything you think > sucks or want, and nobody can disagree with you (yet). So let it all > out. > > -- > Zed A. Shaw > http://zedshaw.com/
On Mon, Sep 6, 2010 at 22:25, Devon Weller <dweller@devonweller.com> wrote: > > What do you think about bringing some Lua into Mongrel2? I'm a big Lua fan here. I've written a fair bit of software with it. In fact, for the past two work days, I've been working on a WSAPI handler for Lua to make web apps easily deployable on Mongrel2. I'm beyond excited about Mongrel2 because I think it will be the first server that offers a performant way to deploy Lua web apps - before we were stuck with FastCGI as our best option. > Perhaps m2sh could be rewritten in Lua. I'm not saying it is easy, but it might ease some of the Python issues. There are sqlite3 bindings at http://luaforge.net/projects/luasqlite/. If Lua were bundled, maybe, but if not then I don't see that as better than leaving it in Python. At least *some* people's OS's already have a reasonable Python installation, but none come with Lua. I don't exactly have a huge amount of experience with Mongrel2 - 2 whole days! So take what I saw with a huge grain of salt. But for me, m2sh being written in Python has had about 0 impact on my work even though I don't know how to program in Python. I just had to install a package or two on Debian, no big deal. I didn't bother installing web.py and some other stuff because I'm mainly interested in Lua right now and don't need to run the Python examples, just read them. If Python is really a problem, then unless Zed or somebody else wants to do it in C, then perhaps it might be best to present m2sh as a "spec" for how to implement a configuration tool for Mongrel2, provide more docs and some kind of cross-platform tests, and help/encourage people to create whatever they want. There are a lot of people who will have some kind of problem with Python, Ruby, Lua, or pretty anything else other than a compiled binary. It seems a shame for hackers familiar with Mongrel2's guts to focus on ancillary things like config tools, when that could probably be done acceptably by less experienced/knowledgable people, if they had the right documentation. > > Also - and this may just be plain crazy - what do you think about allowing users to write custom filters or plugins in Lua? Someone asked about sticky sessions, for example. Instead of writing this filter in C, could a user write it in Lua? > Lighttpd does this, right? As a fan of Lua I would love this, but doesn't this go a bit against the grain of Mongrel's language independence? > Supposedly Lua is fast. I don't know if it is fast enough for a Mongrel2 filter, though. But Lua is a language that is very accessible to web developers. For simple tasks, it is almost identical to javascript. Yeah, particularly LuaJIT is *quite* fast. :) http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=all&d=data&calc=calculate&gpp=on&java=on&luajit=on&v8=on&lua=on&tracemonkey=on&box=1 Regards, Norman
On Mon, Sep 06, 2010 at 11:10:16PM -0300, Norman Clarke wrote: > On Mon, Sep 6, 2010 at 22:25, Devon Weller <dweller@devonweller.com> wrote: > If Python is really a problem, then unless Zed or somebody else wants > to do it in C, then perhaps it might be best to present m2sh as a This is what I'm doing right now. C is the only language everyone doesn't care about and that has no packaging problems. You got gcc, you got all you need. So I'm doing up an m2sh that will do all the basic stuff we've got now, plus the extra stuff and then that gets installed. If people want more then they can look at the Python m2sh to see how to do more. It's actually turning out to not be too hard with the right libraries and such. -- Zed A. Shaw http://zedshaw.com/
On Mon, 6 Sep 2010 21:01:17 -0700, "Zed A. Shaw" <zedshaw@zedshaw.com> wrote: > This is what I'm doing right now. C is the only language everyone > doesn't care about and that has no packaging problems. You got gcc, you > got all you need. So I'm doing up an m2sh that will do all the basic > stuff we've got now, plus the extra stuff and then that gets installed. > If people want more then they can look at the Python m2sh to see how to > do more. > > It's actually turning out to not be too hard with the right libraries > and such. Hm, maybe we could extend this c m2sh to be built with lua support on demand and then have the full power available. I'm thinking of some sort of plugin system, so vanilla c m2sh does the basic tasks and when you compile it with lua support you can make it do all kind of things by adding some lua scripts, if we're going the lua way that is. The base tasks of m2sh can be seen in the current python incarnation, it boils down to sqlite manipulation and starting/stopping processes. Shouldn't be too hard to insert some hooks into the handful of functions so you could add functionality with a lua script calling that hook. That would be purely optional, but still giving users a tool capable of doing everything possible without anyone having to write a third incarnation of m2sh... Thoughts? Greetings, Florian
On Tue, Sep 07, 2010 at 08:34:14AM +0200, Florian Anderiasch wrote: > > On Mon, 6 Sep 2010 21:01:17 -0700, "Zed A. Shaw" <zedshaw@zedshaw.com> > wrote: > > This is what I'm doing right now. C is the only language everyone > > doesn't care about and that has no packaging problems. You got gcc, you > > got all you need. So I'm doing up an m2sh that will do all the basic > > stuff we've got now, plus the extra stuff and then that gets installed. > > If people want more then they can look at the Python m2sh to see how to > > do more. > > > > It's actually turning out to not be too hard with the right libraries > > and such. > > Hm, > maybe we could extend this c m2sh to be built with lua support on demand > and > then have the full power available. That's an idea that's been tossed around too, but first thing I'm doing is making a pure C replacement for what we have in m2sh that does nothing more than that and works. After that we can talk about plugging in other stuff, but honestly I think just writing an m2sh example in lua would be easier. Literally the goal of the C m2sh is that nobody can bitch and whine about dependencies, Python installs, downloading packages, having to use Python, versions of Python, or any other languages. They can have a minimal decent m2sh that works and if people want more they can craft anything else they want. It'll also serve as an example of why this config design works. If I can craft an alternative to the Python m2sh in C then other people can probably do it in their favorite language. Anyway, you should get with jsimmons about the Lua stuff if you're interested in Lua support. -- Zed A. Shaw http://zedshaw.com/
Hi guys. In relation to a binary m2sh written in C, that's something zed and myself have been looking at as well as optionally compiling in lua config support. My work at the moment has been whipping up a quick proof of concept in Lua. Nothing here is set in stone at all, we're just looking at the options. The main reason for all this is the python packaging issue, so we have a simple, self contained configuration solution. BTW as the author of the lua handler, I'd be interested in seeing your wsapi work, and potentially bundling them together. On 7 Sep 2010 12:10, "Norman Clarke" <norman@njclarke.com> wrote: > On Mon, Sep 6, 2010 at 22:25, Devon Weller <dweller@devonweller.com> wrote: >> >> What do you think about bringing some Lua into Mongrel2? > > I'm a big Lua fan here. I've written a fair bit of software with it. > In fact, for the past two work days, I've been working on a WSAPI > handler for Lua to make web apps easily deployable on Mongrel2. I'm > beyond excited about Mongrel2 because I think it will be the first > server that offers a performant way to deploy Lua web apps - before we > were stuck with FastCGI as our best option. > >> Perhaps m2sh could be rewritten in Lua. I'm not saying it is easy, but it might ease some of the Python issues. There are sqlite3 bindings at http://luaforge.net/projects/luasqlite/. > > If Lua were bundled, maybe, but if not then I don't see that as better > than leaving it in Python. At least *some* people's OS's already have > a reasonable Python installation, but none come with Lua. > > I don't exactly have a huge amount of experience with Mongrel2 - 2 > whole days! So take what I saw with a huge grain of salt. But for me, > m2sh being written in Python has had about 0 impact on my work even > though I don't know how to program in Python. I just had to install a > package or two on Debian, no big deal. I didn't bother installing > web.py and some other stuff because I'm mainly interested in Lua right > now and don't need to run the Python examples, just read them. > > If Python is really a problem, then unless Zed or somebody else wants > to do it in C, then perhaps it might be best to present m2sh as a > "spec" for how to implement a configuration tool for Mongrel2, provide > more docs and some kind of cross-platform tests, and help/encourage > people to create whatever they want. There are a lot of people who > will have some kind of problem with Python, Ruby, Lua, or pretty > anything else other than a compiled binary. > > It seems a shame for hackers familiar with Mongrel2's guts to focus on > ancillary things like config tools, when that could probably be done > acceptably by less experienced/knowledgable people, if they had the > right documentation. > >> >> Also - and this may just be plain crazy - what do you think about allowing users to write custom filters or plugins in Lua? Someone asked about sticky sessions, for example. Instead of writing this filter in C, could a user write it in Lua? >> > > Lighttpd does this, right? As a fan of Lua I would love this, but > doesn't this go a bit against the grain of Mongrel's language > independence? > >> Supposedly Lua is fast. I don't know if it is fast enough for a Mongrel2 filter, though. But Lua is a language that is very accessible to web developers. For simple tasks, it is almost identical to javascript. > > Yeah, particularly LuaJIT is *quite* fast. :) > > http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=all&d=data&calc=calculate&gpp=on&java=on&luajit=on&v8=on&lua=on&tracemonkey=on&box=1 > > Regards, > > Norman
On Mon, Sep 6, 2010 at 23:42, joshua simmons <simmons.44@gmail.com> wrote: > BTW as the author of the lua handler, I'd be interested in seeing your wsapi > work, and potentially bundling them together. Sure, I'll let you know as soon as I have something more or less working.
Think would be nice to get a comprehensive test suite for handlers. Testing in a basic request/response kind of way so we can hopefully avoid little issues before they happen. I've also been tossing over the potential of writing the logs to a zmq socket. It would give one a lot of flexibility with what's done with the data. Big issue here and I suppose with all of the logging is making sure the format is very machine readable.
I followed the "deploy" section of the manual step by step and it simply didn't work without LOTS of undocumented tweaking. My number one suggestion would be to simplify this (majorly) to make it super easy for users to get up and running. It's a bit of a barrier to entry right now. I do like the m2sh script, although it should give helpful messages about usage instead of just exception traces when things go wrong. Timestamps in log files are also a must. I'm very excited about this project and hope that I may be able to contribute in some meaningful way later on. Cheers Rich On Mon, Sep 6, 2010 at 8:00 AM, joshua simmons <simmons.44@gmail.com> wrote: > Think would be nice to get a comprehensive test suite for handlers. Testing > in a basic request/response kind of way so we can hopefully avoid little > issues before they happen. > > I've also been tossing over the potential of writing the logs to a zmq > socket. It would give one a lot of flexibility with what's done with the > data. Big issue here and I suppose with all of the logging is making sure > the format is very machine readable. >
On Mon, Sep 06, 2010 at 05:42:16PM +1000, Rich Atkinson wrote: > I followed the "deploy" section of the manual step by step and it simply > didn't work without LOTS of undocumented tweaking. Hey Rich, is there a way you can do an install and then tell us why it took so much undocumented tweaking? It'd help us to document what tweaking you did that's not documented if we knew what tweaking you did that wasn't documented. :-) > I do like the m2sh script, although it should give helpful messages about > usage instead of just exception traces when things go wrong. Yeah, that too. -- Zed A. Shaw http://zedshaw.com/
On Sep 6, 2010, at 3:01 AM, Zed A. Shaw wrote: > On Mon, Sep 06, 2010 at 05:42:16PM +1000, Rich Atkinson wrote: >> I followed the "deploy" section of the manual step by step and it simply >> didn't work without LOTS of undocumented tweaking. > > Hey Rich, is there a way you can do an install and then tell us why it > took so much undocumented tweaking? It'd help us to document what > tweaking you did that's not documented if we knew what tweaking you did > that wasn't documented. :-) > FWIW, on SnowLeopard all I had to adjust was CFLAGS (-I,-L) to accommodate my install location of zmq and PREFIX to set the location where I wanted Mongrel2 to live. Honestly, I didn't look at the docs to see if they covered this case, so I can't speak to the quality of the documentation for this. I had most of the prerequisites installed, since I use a number of those libraries for other projects, so I may not be the ideal test user. Just thought I would contribute my experience as another data point. Kudos on reaching 1.0... -josh
Hey guys, it was on snow leopard. I'll try it again in the next few days and make a list. Unrelated: Installation on that platform was a breeze - (assuming the user has xcode, homebrew, pip): > brew install zeromq > pip install pysqlite > wget http://mongrel2.org/static/downloads/mongrel2-1.0.zip > unzip mongrel2*.zip > cd mongrel2*.zip > make clean all install Rich On Mon, Sep 6, 2010 at 6:01 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote: > On Mon, Sep 06, 2010 at 05:42:16PM +1000, Rich Atkinson wrote: > > I followed the "deploy" section of the manual step by step and it simply > > didn't work without LOTS of undocumented tweaking. > > Hey Rich, is there a way you can do an install and then tell us why it > took so much undocumented tweaking? It'd help us to document what > tweaking you did that's not documented if we knew what tweaking you did > that wasn't documented. :-) > > > I do like the m2sh script, although it should give helpful messages about > > usage instead of just exception traces when things go wrong. > > Yeah, that too. > > -- > Zed A. Shaw > http://zedshaw.com/ >
I used Mac ports on snow leopard and that worked great as well. I installed zmq from source though. On Sep 6, 2010, at 4:11 AM, Rich Atkinson <atkinsonr@gmail.com> wrote: Hey guys, it was on snow leopard. I'll try it again in the next few days and make a list. Unrelated: Installation on that platform was a breeze - (assuming the user has xcode, homebrew, pip): > brew install zeromq > pip install pysqlite > wget http://mongrel2.org/static/downloads/mongrel2-1.0.zip > unzip mongrel2*.zip > cd mongrel2*.zip > make clean all install Rich On Mon, Sep 6, 2010 at 6:01 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote: > On Mon, Sep 06, 2010 at 05:42:16PM +1000, Rich Atkinson wrote: > > I followed the "deploy" section of the manual step by step and it simply > > didn't work without LOTS of undocumented tweaking. > > Hey Rich, is there a way you can do an install and then tell us why it > took so much undocumented tweaking? It'd help us to document what > tweaking you did that's not documented if we knew what tweaking you did > that wasn't documented. :-) > > > I do like the m2sh script, although it should give helpful messages about > > usage instead of just exception traces when things go wrong. > > Yeah, that too. > > -- > Zed A. Shaw > http://zedshaw.com/ >
Odd. I followed the manual to the letter as part of my editing and had no problems. Would you mind listing some of the tweaking you did? Tordek -----Original message----- From: Rich Atkinson <atkinsonr@gmail.com> To: mongrel2@librelist.com Sent: Mon, 06 Sep 2010 07:42:37 GMT+00:00 Subject: Re: [mongrel2] What Sucks In 1.0 I followed the "deploy" section of the manual step by step and it simply didn't work without LOTS of undocumented tweaking. My number one suggestion would be to simplify this (majorly) to make it super easy for users to get up and running. It's a bit of a barrier to entry right now. I do like the m2sh script, although it should give helpful messages about usage instead of just exception traces when things go wrong. Timestamps in log files are also a must. I'm very excited about this project and hope that I may be able to contribute in some meaningful way later on. Cheers Rich On Mon, Sep 6, 2010 at 8:00 AM, joshua simmons <simmons.44@gmail.com> wrote: > Think would be nice to get a comprehensive test suite for handlers. Testing > in a basic request/response kind of way so we can hopefully avoid little > issues before they happen. > > I've also been tossing over the potential of writing the logs to a zmq > socket. It would give one a lot of flexibility with what's done with the > data. Big issue here and I suppose with all of the logging is making sure > the format is very machine readable. >
Agree with Tordek, I'm on ArchLinux and the instructions in the manual work perfectly. Which distributive are you using? On Mon, Sep 6, 2010 at 1:49 PM, Tordek <kedrot@gmail.com> wrote: > Odd. I followed the manual to the letter as part of my editing and had no > problems. Would you mind listing some of the tweaking you did? > > Tordek > > -----Original message----- > > From: Rich Atkinson <atkinsonr@gmail.com> > To: mongrel2@librelist.com > Sent: Mon, 06 Sep 2010 07:42:37 GMT+00:00 > Subject: Re: [mongrel2] What Sucks In 1.0 > > I followed the "deploy" section of the manual step by step and it simply > didn't work without LOTS of undocumented tweaking. > My number one suggestion would be to simplify this (majorly) to make it > super easy for users to get up and running. It's a bit of a barrier to entry > right now. > I do like the m2sh script, although it should give helpful messages about > usage instead of just exception traces when things go wrong. > Timestamps in log files are also a must. > I'm very excited about this project and hope that I may be able to > contribute in some meaningful way later on. > Cheers > Rich > > > On Mon, Sep 6, 2010 at 8:00 AM, joshua simmons <simmons.44@gmail.com> wrote: >> >> Think would be nice to get a comprehensive test suite for handlers. >> Testing in a basic request/response kind of way so we can hopefully avoid >> little issues before they happen. >> >> I've also been tossing over the potential of writing the logs to a zmq >> socket. It would give one a lot of flexibility with what's done with the >> data. Big issue here and I suppose with all of the logging is making sure >> the format is very machine readable. > -- Good luck, Pasha Bastov
- HTTP and SSL/TLS - Tunnels - Crypto 3DES, Data integrity: SHA-1, Key Exchange DH Group2 (1024bit), Perfect Forward Secrecy - IPSec - VPN - Continuous integration charts per day ... - Integrate Fossil scripts? to automate remote deployment sync, build and start new in bg hand over req, old phase out and down ... - Use of field "Subsystem" in Fossil tickets /xraid
On Sun, Sep 5, 2010 at 12:22 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:
> * Python m2py needs WSGI bad. Real bad.
I've fixed all the bugs I could find, and added a first pass at Django
integration.
http://github.com/timothyfitz/mongrel2_wsgi
Install, add mongrel2_wsgi to your INSTALLED_APPS in Django, and then just
run python manage.py m2wsgi.
Enjoy, patches, bug repros or bug reports much appreciated!
Timothy
On Sun, Sep 05, 2010 at 12:38:24PM -0700, Timothy Fitz wrote: > On Sun, Sep 5, 2010 at 12:22 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote: > > > * Python m2py needs WSGI bad. Real bad. > > > I've fixed all the bugs I could find, and added a first pass at Django > integration. Cooooool, I shall rip this off and try to roll it into the main distro. Thanks. -- Zed A. Shaw http://zedshaw.com/