librelist archives

« back to archive

Performance tuning.

Performance tuning.

From:
joshua simmons
Date:
2011-03-20 @ 11:58
Heya,

Neopallium and I have been doing a bit of performance work on mongrel2 trunk
and it's been getting some pretty good results in my testing, however I need
people to have a play and test and make sure I haven't broken anything.

In my completely synthetic mongrel2-lua benchmark which echos the request
headers back at the browser, I've seen an increase from ~15k req/s to ~20k
reqs/s on my machine, hopefully those more CPU bound should see even bigger
performance gains. Very interested in your results.

The good news is there's still plenty of work to be done too!

On a related note I changed up the hashing function for the request headers,
this seems to be a good change but adt is determining the hash bit size at
runtime and I'm simply applying the constants for a 64bit hash. I'm thinking
setting the hash_val_t_bit to 64 would be a good idea, as well as fixing
hash_val_t to a uint64_t (or even 32 and 32). However I'm not sure if I
should be bringing dependencies on c99 in (to get the uintx_t types easily).
Feedback appreciated.

Thanks,
Josh.

Re: [mongrel2] Performance tuning.

From:
Zed A. Shaw
Date:
2011-03-20 @ 13:17
On Sun, Mar 20, 2011 at 10:58:32PM +1100, joshua simmons wrote:
> Heya,
> 
> Neopallium and I have been doing a bit of performance work on mongrel2 trunk
> and it's been getting some pretty good results in my testing, however I need
> people to have a play and test and make sure I haven't broken anything.

This is working for me too.  I've got it running on zedshaw.com and it
seems to be holding up.  I'll toss something on twitter and have the
bots and followers hit it some more

> The good news is there's still plenty of work to be done too!

Cool, let me know of Neopallium wants commit access.

> On a related note I changed up the hashing function for the request headers,
> this seems to be a good change but adt is determining the hash bit size at
> runtime and I'm simply applying the constants for a 64bit hash. I'm thinking
> setting the hash_val_t_bit to 64 would be a good idea, as well as fixing
> hash_val_t to a uint64_t (or even 32 and 32). However I'm not sure if I
> should be bringing dependencies on c99 in (to get the uintx_t types easily).
> Feedback appreciated.

That's already being done all over the place, so totally alright.  Only
limitation would be making sure that 64bit value would work reliably on
all 32 bit platforms.  Otherwise, C99 is the standard I shoot for and I
don't support non-c99 compilers at all.

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] Performance tuning.

From:
joshua simmons
Date:
2011-03-21 @ 11:45
I changed it to a 32 bit hash and gained some more performance too.

I have no idea if this is going to have an affect given our use case
(although I presume not), so if anyone who actually knows what they're
talking about when it comes to string hashing would lend their input that'd
be great. Failing that testing will do.

On my machine coupled with luajit and trunk mongrel2-lua I'm now up to a
handy 22k req/s with the synthetic benchmark. It'd be interesting if we had
a more comprehensive set of benchmarks to compare with.

On Mon, Mar 21, 2011 at 12:17 AM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:

> On Sun, Mar 20, 2011 at 10:58:32PM +1100, joshua simmons wrote:
> > Heya,
> >
> > Neopallium and I have been doing a bit of performance work on mongrel2
> trunk
> > and it's been getting some pretty good results in my testing, however I
> need
> > people to have a play and test and make sure I haven't broken anything.
>
> This is working for me too.  I've got it running on zedshaw.com and it
> seems to be holding up.  I'll toss something on twitter and have the
> bots and followers hit it some more
>
> > The good news is there's still plenty of work to be done too!
>
> Cool, let me know of Neopallium wants commit access.
>
> > On a related note I changed up the hashing function for the request
> headers,
> > this seems to be a good change but adt is determining the hash bit size
> at
> > runtime and I'm simply applying the constants for a 64bit hash. I'm
> thinking
> > setting the hash_val_t_bit to 64 would be a good idea, as well as fixing
> > hash_val_t to a uint64_t (or even 32 and 32). However I'm not sure if I
> > should be bringing dependencies on c99 in (to get the uintx_t types
> easily).
> > Feedback appreciated.
>
> That's already being done all over the place, so totally alright.  Only
> limitation would be making sure that 64bit value would work reliably on
> all 32 bit platforms.  Otherwise, C99 is the standard I shoot for and I
> don't support non-c99 compilers at all.
>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

Re: [mongrel2] Performance tuning.

From:
Zed A. Shaw
Date:
2011-03-21 @ 16:38
On Mon, Mar 21, 2011 at 10:45:28PM +1100, joshua simmons wrote:
> On my machine coupled with luajit and trunk mongrel2-lua I'm now up to a
> handy 22k req/s with the synthetic benchmark. It'd be interesting if we had
> a more comprehensive set of benchmarks to compare with.

Interesting.  Hey, can you write up or link me to some instructions for
getting luazmq working in luajit?

-- 
Zed A. Shaw
http://zedshaw.com/

Re: [mongrel2] Performance tuning.

From:
joshua simmons
Date:
2011-03-21 @ 21:08
It works out of the box, luajit being a drop in replacement for standard
Lua.

I'm not sure though if I'm using Neopallium's FFI enhanced version or not.
It's over here https://github.com/Neopallium/lua-zmq

Still, I'll have a look tonight if I get a chance.

On Tue, Mar 22, 2011 at 3:38 AM, Zed A. Shaw <zedshaw@zedshaw.com> wrote:

> On Mon, Mar 21, 2011 at 10:45:28PM +1100, joshua simmons wrote:
> > On my machine coupled with luajit and trunk mongrel2-lua I'm now up to a
> > handy 22k req/s with the synthetic benchmark. It'd be interesting if we
> had
> > a more comprehensive set of benchmarks to compare with.
>
> Interesting.  Hey, can you write up or link me to some instructions for
> getting luazmq working in luajit?
>
> --
> Zed A. Shaw
> http://zedshaw.com/
>