Hey everyone, I just put up an initial version of the new request tracking gear in Mongrel2. The changes to src/register.c will make Mongrel2 use a connection ID (that's sent to handlers) that is a uint32 sized perpetually increasing id, so it should not collide practically. I mean, there may be someone doing 4.29billion connections in a few seconds on a single mongrel2 but hopefully not. You can look in src/adt/radixmap.c for the algorithm being used. I was going to just use a heap, but that's boring so I used a Radix sort, which has some interesting properties for this problem. I then spent a few days tuning it based on some randomized sampling and a few tricks I know. If you find bugs in the radixmap.c code, or can make it faster then let me know. The next thing I'll be doing is: 1. Avoiding making connection IDs until the request really is going to need it. 2. Making the number of simultaneous requests Mongrel2 can hold configurable. 3. Start tuning the mongrel2 code and doing code reviews. After that, I think it'll be good for a release. Is there anything people think should go in 1.8.0? -- Zed A. Shaw http://zedshaw.com/
Timestamps for log entries in access.log and error.log would be super helpful. ✈ Matt On Jul 28, 2011, at 23:44 , Zed A. Shaw wrote: > After that, I think it'll be good for a release. Is there anything > people think should go in 1.8.0?
Any issues with backwards compatibility? ✈ Matt On Jul 28, 2011, at 23:44 , Zed A. Shaw wrote: > Hey everyone, > > I just put up an initial version of the new request tracking gear in > Mongrel2. The changes to src/register.c will make Mongrel2 use a > connection ID (that's sent to handlers) that is a uint32 sized > perpetually increasing id, so it should not collide practically. I > mean, there may be someone doing 4.29billion connections in a few > seconds on a single mongrel2 but hopefully not. > > You can look in src/adt/radixmap.c for the algorithm being > used. I was going to just use a heap, but that's boring so I used a > Radix sort, which has some interesting properties for this problem. I > then spent a few days tuning it based on some randomized sampling and a > few tricks I know. > > If you find bugs in the radixmap.c code, or can make it faster then let > me know. > > The next thing I'll be doing is: > > 1. Avoiding making connection IDs until the request really is going to > need it. > 2. Making the number of simultaneous requests Mongrel2 can hold > configurable. > 3. Start tuning the mongrel2 code and doing code reviews. > > After that, I think it'll be good for a release. Is there anything > people think should go in 1.8.0? > > -- > Zed A. Shaw > http://zedshaw.com/
The static files must be put in mongrel2's chroot directory now, if those static file directories can be a soft link to another directory outside of mongrel2's chroot directory, it will be better. Because my projects of bamboo are always outside mongrel2's chroot directory, but the images, css, js have to be put in mongrel2's chroot directory yet, this doesn't very convenient to manage. Any suggestion? On Fri, Jul 29, 2011 at 2:44 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote: > Hey everyone, > > I just put up an initial version of the new request tracking gear in > Mongrel2. The changes to src/register.c will make Mongrel2 use a > connection ID (that's sent to handlers) that is a uint32 sized > perpetually increasing id, so it should not collide practically. I > mean, there may be someone doing 4.29billion connections in a few > seconds on a single mongrel2 but hopefully not. > > You can look in src/adt/radixmap.c for the algorithm being > used. I was going to just use a heap, but that's boring so I used a > Radix sort, which has some interesting properties for this problem. I > then spent a few days tuning it based on some randomized sampling and a > few tricks I know. > > If you find bugs in the radixmap.c code, or can make it faster then let > me know. > > The next thing I'll be doing is: > > 1. Avoiding making connection IDs until the request really is going to > need it. > 2. Making the number of simultaneous requests Mongrel2 can hold > configurable. > 3. Start tuning the mongrel2 code and doing code reviews. > > After that, I think it'll be good for a release. Is there anything > people think should go in 1.8.0? > > -- > Zed A. Shaw > http://zedshaw.com/ > -- Nothing is impossible.
On Fri, Jul 29, 2011 at 03:09:46PM +0800, Tang Daogang wrote: > Because my projects of bamboo are always outside mongrel2's chroot > directory, but the images, css, js have to be put in mongrel2's chroot > directory yet, this doesn't very convenient to manage. Huh? I do this all the time. Mongrel2 is chroot at /var/www/ then every site has a dir in there like /var/www/mongrel2.org and then I'll symlink to that dir and do the setup. I think you're just doing it backwards. Instead of linking inside the chroot to a directory with assets in your app, link your directory of assets in your app to the chroot. For example, here's guicheck.com's setup: drwxr-xr-x 4 4096 Jul 15 09:15 app drwxr-xr-x 4 4096 Jul 17 16:12 assets drwxr-xr-x 2 4096 Jul 16 12:13 conf drwxr-xr-x 2 4096 Jul 15 14:47 logs drwxr-xr-x 2 4096 Jul 23 12:51 model lrwxrwxrwx 1 21 Jul 17 16:38 output -> /var/www/guicheck.com drwxr-xr-x 2 4096 Jul 23 19:00 run drwxr-xr-x 4 4096 Jul 17 19:37 src drwxr-xr-x 3 4096 Jul 10 17:04 tests drwxr-xr-x 2 4096 Jul 15 09:54 tmp drwxr-xr-x 2 4096 Jul 25 10:11 views Then the mongrel2 chroot is /var/www/, and I have a Makefile that I can use to "build" things into the output directory. The build will do stuff like copy assets, static site generation, etc. and then the app writes anything static it makes into there. The Tir app then just runs from this directory and everything works fine. This also has the added benefit that the mongrel2 process, which is external to the world, is chroot and can't get at the application. So the only thing an attacker could get at is static files mongrel2 has, rather than all the app code, logs, user databases, etc. Basically, invert your idea and it'll be way easier. -- Zed A. Shaw http://zedshaw.com/
I use mount --bind to mount statics into the chroot of my deployments, since symlinks can never work in a chroot and you cant hardlink a dir. On Jul 29, 2011, at 3:09 AM, Tang Daogang <daogangtang@gmail.com> wrote: > The static files must be put in mongrel2's chroot directory now, if those static file directories can be a soft link to another directory outside of mongrel2's chroot directory, it will be better. > > Because my projects of bamboo are always outside mongrel2's chroot directory, but the images, css, js have to be put in mongrel2's chroot directory yet, this doesn't very convenient to manage. > > Any suggestion? > > On Fri, Jul 29, 2011 at 2:44 PM, Zed A. Shaw <zedshaw@zedshaw.com> wrote: > Hey everyone, > > I just put up an initial version of the new request tracking gear in > Mongrel2. The changes to src/register.c will make Mongrel2 use a > connection ID (that's sent to handlers) that is a uint32 sized > perpetually increasing id, so it should not collide practically. I > mean, there may be someone doing 4.29billion connections in a few > seconds on a single mongrel2 but hopefully not. > > You can look in src/adt/radixmap.c for the algorithm being > used. I was going to just use a heap, but that's boring so I used a > Radix sort, which has some interesting properties for this problem. I > then spent a few days tuning it based on some randomized sampling and a > few tricks I know. > > If you find bugs in the radixmap.c code, or can make it faster then let > me know. > > The next thing I'll be doing is: > > 1. Avoiding making connection IDs until the request really is going to > need it. > 2. Making the number of simultaneous requests Mongrel2 can hold > configurable. > 3. Start tuning the mongrel2 code and doing code reviews. > > After that, I think it'll be good for a release. Is there anything > people think should go in 1.8.0? > > -- > Zed A. Shaw > http://zedshaw.com/ > > > > -- > Nothing is impossible. >
Oh, Good idea! Thanks for your suggestion. Wonderful ! On Fri, Jul 29, 2011 at 8:54 PM, Yaroslav Shirokov <slava@cyberwang.net>wrote: > I use mount --bind to mount statics into the chroot of my deployments, > since symlinks can never work in a chroot and you cant hardlink a dir. > > On Jul 29, 2011, at 3:09 AM, Tang Daogang <daogangtang@gmail.com> wrote: > > The static files must be put in mongrel2's chroot directory now, if those > static file directories can be a soft link to another directory outside of > mongrel2's chroot directory, it will be better. > > Because my projects of bamboo are always outside mongrel2's chroot > directory, but the images, css, js have to be put in mongrel2's chroot > directory yet, this doesn't very convenient to manage. > > Any suggestion? > > On Fri, Jul 29, 2011 at 2:44 PM, Zed A. Shaw < <zedshaw@zedshaw.com> > zedshaw@zedshaw.com> wrote: > >> Hey everyone, >> >> I just put up an initial version of the new request tracking gear in >> Mongrel2. The changes to src/register.c will make Mongrel2 use a >> connection ID (that's sent to handlers) that is a uint32 sized >> perpetually increasing id, so it should not collide practically. I >> mean, there may be someone doing 4.29billion connections in a few >> seconds on a single mongrel2 but hopefully not. >> >> You can look in src/adt/radixmap.c for the algorithm being >> used. I was going to just use a heap, but that's boring so I used a >> Radix sort, which has some interesting properties for this problem. I >> then spent a few days tuning it based on some randomized sampling and a >> few tricks I know. >> >> If you find bugs in the radixmap.c code, or can make it faster then let >> me know. >> >> The next thing I'll be doing is: >> >> 1. Avoiding making connection IDs until the request really is going to >> need it. >> 2. Making the number of simultaneous requests Mongrel2 can hold >> configurable. >> 3. Start tuning the mongrel2 code and doing code reviews. >> >> After that, I think it'll be good for a release. Is there anything >> people think should go in 1.8.0? >> >> -- >> Zed A. Shaw >> <http://zedshaw.com/>http://zedshaw.com/ >> > > > > -- > Nothing is impossible. > > -- Nothing is impossible.