What's the algorithm for calculating cowbells? -- Chad
On Wed, Dec 30, 2009 at 09:43:13PM -0700, Chad Woolley wrote: > What's the algorithm for calculating cowbells? select avg(mean) as rating, avg(sd) as rating_sd, other_id, player_id, round_id from statistic, submission where other_id=submission.id and other_type='submission' and round_id = $rnd.id group by other_id, player_id, round_id order by round_id desc, rating desc, rating_sd asc; That's the query. It's just the average of all the means times 1000 so it's not a fractional number. This produces this table: rating | rating_sd | other_id | player_id | round_id -------------------+-------------------+----------+-----------+-- 1.19444445272287 | 0.498557180166245 | 81 | 181 | 71 1.1363636503617 | 0.54525621732076 | 73 | 39 | 71 1.07575759788354 | 0.495001117388407 | 79 | 199 | 71 1.00000000745058 | 0.542712951699893 | 80 | 1 | 71 0.972222242504358 | 0.431989908218384 | 78 | 120 | 71 0.962962968895833 | 0.494796648621559 | 77 | 40 | 71 0.928571457664172 | 0.49314454694589 | 82 | 20 | 71 If we pop the raw scores from that first one (submission id 81) we get: >>> (0.333 + 0.5 + 0.666 + 1 + 4 + 0.666)/6 1.1941666666666668 And finally if you mutliply by 1000 you get: 1194 cowbells. Notice also that the ranking is influenced by the rating_sd, which is how tightly or widely people's opinions of you range. So people with a lower sd get ranked higher as being more consistent. Keep in mind that I gotta redo round 6 because of the bogus votes from Ram, which will happen tomorrow when I write round 7 and announce round 6 done on the fret show. -- Zed A. Shaw http://zedshaw.com/
On Thu, Dec 31, 2009 at 1:05 AM, Zed A. Shaw <zedshaw@zedshaw.com> wrote: > On Wed, Dec 30, 2009 at 09:43:13PM -0700, Chad Woolley wrote: >> What's the algorithm for calculating cowbells? > > select avg(mean) as rating, avg(sd) as rating_sd, other_id, > player_id, round_id > from statistic, submission where other_id=submission.id and > other_type='submission' > and round_id = $rnd.id > group by other_id, player_id, round_id order by round_id > desc, rating desc, rating_sd asc; > > That's the query. It's just the average of all the means times 1000 so > it's not a fractional number. > > This produces this table: > rating | rating_sd | other_id | player_id | round_id > -------------------+-------------------+----------+-----------+-- > 1.19444445272287 | 0.498557180166245 | 81 | 181 | 71 > 1.1363636503617 | 0.54525621732076 | 73 | 39 | 71 > 1.07575759788354 | 0.495001117388407 | 79 | 199 | 71 > 1.00000000745058 | 0.542712951699893 | 80 | 1 | 71 > 0.972222242504358 | 0.431989908218384 | 78 | 120 | 71 > 0.962962968895833 | 0.494796648621559 | 77 | 40 | 71 > 0.928571457664172 | 0.49314454694589 | 82 | 20 | 71 > > If we pop the raw scores from that first one (submission id 81) we get: > >>>> (0.333 + 0.5 + 0.666 + 1 + 4 + 0.666)/6 > 1.1941666666666668 > > And finally if you mutliply by 1000 you get: > > 1194 cowbells. Still confused. This calculation is for a single round? If so: 1. How were the raw scores calculated (where did 0.333) come from? 2. How does the standard deviation factor in (above looks like just the average of the raws)? How does a lower SD result in a higher score? 3. What is the standard deviation of? The raw scores? If so, how can there be multiple SDs per round or one SD per raw score (and why would they be averaged)? Or, if this calculation isn't for a single round, what is it for? Sorry if I'm obtuse. I've always thought I was the inspiration for your "programmers must learn statistics of I will kill them" article... -- Chad
On Thu, Dec 31, 2009 at 04:01:36AM -0700, Chad Woolley wrote: > Still confused. This calculation is for a single round? If so: http://zedshaw.com/blog/2009-11-6.html Explains the math behind it. > 1. How were the raw scores calculated (where did 0.333) come from? It's from a rolling statistic of the five qualitative elements and the overall rating. It's done so that it can be calculated as each person submits their rating, rather than having to be done repeatedly over all the comments each time. See the blog post. > 2. How does the standard deviation factor in (above looks like just > the average of the raws)? How does a lower SD result in a higher > score? Nope, it's the average of the standard deviations for all of the measurements. So it's average of accuracy, tone, speed, overall_rating's standard deviation. Think of it as a "meta statistic". Each stat has a set of numeric elements, which were derived from people voting. Now if I then take all the stats of your submssion, and get the mean of each numeric element, then I've got a normalized meta stat I can use to say where you stand. In this case, the sd is more like meta-sd. It's the mean of all your sd ratings so far, or it could be the mean of the sd of all your ratings ever. > 3. What is the standard deviation of? The raw scores? If so, how can > there be multiple SDs per round or one SD per raw score (and why would > they be averaged)? There are six stats you can have on a submission. Each stat has variables n, sum, mean, sd, min, max. There's only one set of these for each submission, and as people vote, these numbers are recalulated using their vote. So for each submission that's 6 sd (one for each stat). The average sd is then what I showed you before in the other email and in that blog post. It's just the sum of these std.devs / 6. > Or, if this calculation isn't for a single round, what is it for? It's for a single round and just rolls up all your stats into a nice easy to understand number. But, it can also be used for the "best player" stats. Instead of doing it for one round, just do the average for all the rounds and you've got a meta-statistic that ranks you among other players. I could also do it for the whole site, by averaging all the stats I could see how good all the players are. > Sorry if I'm obtuse. I've always thought I was the inspiration for > your "programmers must learn statistics of I will kill them" > article... No problem, but you should read that blog post and read more about mean, std.dev, and the mean of means, and mean of std.devs and why they're important. Hint: binary votes on accuracy aren't actually normally distributed, they're logit, but if I average a bunch of them together they become normally distributed. -- Zed A. Shaw http://zedshaw.com/