librelist archives

« back to archive

On the lack of participants and the Zealous Teacher Paradox

On the lack of participants and the Zealous Teacher Paradox

From:
Peter Minten
Date:
2014-02-10 @ 18:08

Hi all,

I'd like to raise a few issues about exercism.

Let me first define a few terms:

"student": Someone who sends in submissions but does not nitpick.
"teacher": Someone who nitpicks but does not send in submissions.
"participant: Someone who sends in submissions and also nitpicks.

I've noticed a paradox with regard to teachers, something I call the
Zealous Teacher Paradox: if you have a teacher who is knowledgeable and
who responds to every submission or comment within a reasonable time
frame the track will suffer as a result.

Here's my logic:

1. Teacher gathers a reputation for being knowledgeable.
2. Students and participants start to defer to the teacher.
3. Participants avoid nitpicking because they know the teacher will soon
leave a good comment.
4. The teacher becomes effectively the only nitpicker of the track. The
participants don't bother checking the track any more.
5. The teacher gets bored or has to reallocate time and no longer keeps
up with the track.

Currently this is happening with the Elixir track, which is at stage 4.
I notice that nitpicking bob, anagram, even list-ops over and over again
is becoming quite boring and that the quality of my nits is decreasing.
But I seem to have scared away most other nitpickers so the progress of
students on the track seems to depend on me.

Haskell seemed to be another candidate for this problem due to it's
reliance on @etrepum, but luckily participants @qnikst and @mfoda seem
to have picked it up so that the track is in good shape.

The solution to the Zealous Teacher Paradox is I think two fold: it
requires the teacher to realize that it's sometimes better to be less
responsive and it requires students to step up and become participants.

Unfortunately I get the impression that the student->participant
conversion ratio has become worse lately. I suspect it's related to the
change that you can have multiple submissions pending at the same time.
The impression I get is that people don't so much send in more
submissions before leaving the site than that they send in the same
amount in a shorter time by sending in 2 or 3 at once.

The conversion of students to participants on the other hand seems to be
fuelled by a sense of familiarity with the site and the nitpickers on a
track. The best participants I've seen are those that I've had
conversations with over a long series of submissions. This might be
because interacting with someone over a longer period of time creates
somewhat of a sense of "belonging to the exercism club" (for want of a
better term) and also because over several exercises with few nits
people get a feeling that they indeed know the language well enough to
help others.

There are no obvious solutions to converting more students into
participants, but it's important that we think about this because in the
end exercism needs to encourage enough people to become participants in
order to grow.

Greetings,

Peter

Re: [exercism] On the lack of participants and the Zealous Teacher Paradox

From:
Mark IJbema
Date:
2014-02-10 @ 22:35
Some time ago I went to phase 1-5 on the ruby track, so I want to second
this sentiment (though I think the track itself will be less affected,
since it has way more teachers).

I thought a solution might be to only do old submissions when you're a
teacher. Only if a submission lays around for more then a week, nitpick it.
I think this might help the system get in balance.

I also have impression on the lessened conversion since opening up all
exercises. I have no hard data to support this impression though.


On Mon, Feb 10, 2014 at 7:08 PM, Peter Minten <peter@pminten.nl> wrote:

>
>
> Hi all,
>
> I'd like to raise a few issues about exercism.
>
> Let me first define a few terms:
>
> "student": Someone who sends in submissions but does not nitpick.
> "teacher": Someone who nitpicks but does not send in submissions.
> "participant: Someone who sends in submissions and also nitpicks.
>
> I've noticed a paradox with regard to teachers, something I call the
> Zealous Teacher Paradox: if you have a teacher who is knowledgeable and
> who responds to every submission or comment within a reasonable time
> frame the track will suffer as a result.
>
> Here's my logic:
>
> 1. Teacher gathers a reputation for being knowledgeable.
> 2. Students and participants start to defer to the teacher.
> 3. Participants avoid nitpicking because they know the teacher will soon
> leave a good comment.
> 4. The teacher becomes effectively the only nitpicker of the track. The
> participants don't bother checking the track any more.
> 5. The teacher gets bored or has to reallocate time and no longer keeps
> up with the track.
>
> Currently this is happening with the Elixir track, which is at stage 4.
> I notice that nitpicking bob, anagram, even list-ops over and over again
> is becoming quite boring and that the quality of my nits is decreasing.
> But I seem to have scared away most other nitpickers so the progress of
> students on the track seems to depend on me.
>
> Haskell seemed to be another candidate for this problem due to it's
> reliance on @etrepum, but luckily participants @qnikst and @mfoda seem
> to have picked it up so that the track is in good shape.
>
> The solution to the Zealous Teacher Paradox is I think two fold: it
> requires the teacher to realize that it's sometimes better to be less
> responsive and it requires students to step up and become participants.
>
> Unfortunately I get the impression that the student->participant
> conversion ratio has become worse lately. I suspect it's related to the
> change that you can have multiple submissions pending at the same time.
> The impression I get is that people don't so much send in more
> submissions before leaving the site than that they send in the same
> amount in a shorter time by sending in 2 or 3 at once.
>
> The conversion of students to participants on the other hand seems to be
> fuelled by a sense of familiarity with the site and the nitpickers on a
> track. The best participants I've seen are those that I've had
> conversations with over a long series of submissions. This might be
> because interacting with someone over a longer period of time creates
> somewhat of a sense of "belonging to the exercism club" (for want of a
> better term) and also because over several exercises with few nits
> people get a feeling that they indeed know the language well enough to
> help others.
>
> There are no obvious solutions to converting more students into
> participants, but it's important that we think about this because in the
> end exercism needs to encourage enough people to become participants in
> order to grow.
>
> Greetings,
>
> Peter
>
>

Re: [exercism] On the lack of participants and the Zealous Teacher Paradox

From:
David Rueck
Date:
2014-02-10 @ 23:31
I agree that the ability to submit multiple exercises for the same language
at once could be contributing to people not nitpicking as much. When you're
waiting for feedback on one exercise and can't do anything until you've
gotten it, it leads a little more naturally into looking for other ways to
participate, including helping other people progress. It also definitely
contributes to a ton more work for the people who are committed to
nitpicking and makes it feel more one-sided, like there are "students" who
are submitting many many exercises and far fewer "teachers" who are trying
to wade through the giant list of multiple submissions per student.

This is sort of off topic, but related to the multiple submissions at once,
I've seen people make the same "mistake" in multiple exercises because they
submitted them all at once without first receiving feedback on the first
one. If you submit exercises one at a time, then you have a chance to learn
from the feedback on the first one before submitting the next one. That
might not be an issue for everyone using the site, but just something I've
noticed.

I replied to Peter directly, but I've been guilty of not nitpicking as much
on the Elixir track as I should because he's so on top of it (and because
I've been trying to help out on the VERY bogged down JavaScript track), but
I will do my best to try to at least get a 1:1 or 1:2+ ratio of submissions
to nitpicks on the Elixir track and try to help out there more.

-David


On Mon, Feb 10, 2014 at 2:35 PM, Mark IJbema <markijbema@gmail.com> wrote:

> Some time ago I went to phase 1-5 on the ruby track, so I want to second
> this sentiment (though I think the track itself will be less affected,
> since it has way more teachers).
>
> I thought a solution might be to only do old submissions when you're a
> teacher. Only if a submission lays around for more then a week, nitpick it.
> I think this might help the system get in balance.
>
> I also have impression on the lessened conversion since opening up all
> exercises. I have no hard data to support this impression though.
>
>
> On Mon, Feb 10, 2014 at 7:08 PM, Peter Minten <peter@pminten.nl> wrote:
>
>>
>>
>> Hi all,
>>
>> I'd like to raise a few issues about exercism.
>>
>> Let me first define a few terms:
>>
>> "student": Someone who sends in submissions but does not nitpick.
>> "teacher": Someone who nitpicks but does not send in submissions.
>> "participant: Someone who sends in submissions and also nitpicks.
>>
>> I've noticed a paradox with regard to teachers, something I call the
>> Zealous Teacher Paradox: if you have a teacher who is knowledgeable and
>> who responds to every submission or comment within a reasonable time
>> frame the track will suffer as a result.
>>
>> Here's my logic:
>>
>> 1. Teacher gathers a reputation for being knowledgeable.
>> 2. Students and participants start to defer to the teacher.
>> 3. Participants avoid nitpicking because they know the teacher will soon
>> leave a good comment.
>> 4. The teacher becomes effectively the only nitpicker of the track. The
>> participants don't bother checking the track any more.
>> 5. The teacher gets bored or has to reallocate time and no longer keeps
>> up with the track.
>>
>> Currently this is happening with the Elixir track, which is at stage 4.
>> I notice that nitpicking bob, anagram, even list-ops over and over again
>> is becoming quite boring and that the quality of my nits is decreasing.
>> But I seem to have scared away most other nitpickers so the progress of
>> students on the track seems to depend on me.
>>
>> Haskell seemed to be another candidate for this problem due to it's
>> reliance on @etrepum, but luckily participants @qnikst and @mfoda seem
>> to have picked it up so that the track is in good shape.
>>
>> The solution to the Zealous Teacher Paradox is I think two fold: it
>> requires the teacher to realize that it's sometimes better to be less
>> responsive and it requires students to step up and become participants.
>>
>> Unfortunately I get the impression that the student->participant
>> conversion ratio has become worse lately. I suspect it's related to the
>> change that you can have multiple submissions pending at the same time.
>> The impression I get is that people don't so much send in more
>> submissions before leaving the site than that they send in the same
>> amount in a shorter time by sending in 2 or 3 at once.
>>
>> The conversion of students to participants on the other hand seems to be
>> fuelled by a sense of familiarity with the site and the nitpickers on a
>> track. The best participants I've seen are those that I've had
>> conversations with over a long series of submissions. This might be
>> because interacting with someone over a longer period of time creates
>> somewhat of a sense of "belonging to the exercism club" (for want of a
>> better term) and also because over several exercises with few nits
>> people get a feeling that they indeed know the language well enough to
>> help others.
>>
>> There are no obvious solutions to converting more students into
>> participants, but it's important that we think about this because in the
>> end exercism needs to encourage enough people to become participants in
>> order to grow.
>>
>> Greetings,
>>
>> Peter
>>
>>
>

Re: [exercism] On the lack of participants and the Zealous Teacher Paradox

From:
Katrina Owen
Date:
2014-02-11 @ 02:55
On 02/10, David Rueck wrote:
> I agree that the ability to submit multiple exercises for the same language
> at once could be contributing to people not nitpicking as much.

I think you might be right, and at the same time, I don't yet want to
switch it back.

I'm hoping to find some sort of middle ground where people can work on
several things at once, but where they maybe don't go hogwild.

> It also definitely
> contributes to a ton more work for the people who are committed to
> nitpicking and makes it feel more one-sided, like there are "students" who
> are submitting many many exercises and far fewer "teachers" who are trying
> to wade through the giant list of multiple submissions per student.

Yeah. There are "students" that I've stopped nitpicking, because it
I get a bit grumpy that they've submitted 70+ exercises without ever
nitpicking once.

> This is sort of off topic, but related to the multiple submissions at once,
> I've seen people make the same "mistake" in multiple exercises because they
> submitted them all at once without first receiving feedback on the first
> one.

Agreed, that is why I originally had limited it.

The thing that I don't seem to be able to convey to people is that
they're likely to learn *more* from reading and dissecting other
people's code, and trying to come up with a coherent way of expressing
your thoughts about that code than by solving an exercise and moving on.

Katrina

Re: [exercism] On the lack of participants and the Zealous Teacher Paradox

From:
Mark IJbema
Date:
2014-02-11 @ 06:21
On Tue, Feb 11, 2014 at 3:55 AM, Katrina Owen <_@kytrinyx.com> wrote:
>
> The thing that I don't seem to be able to convey to people is that
> they're likely to learn *more* from reading and dissecting other
> people's code, and trying to come up with a coherent way of expressing
> your thoughts about that code than by solving an exercise and moving on.
>

Because that's only a thing you learn by doing. It's not logical that you
learn from teaching. You already know the subject (clearly, since you are
teaching), so you're only using your time, to help others. How would this
help you? Only after you have teached for a while, you will notice this
(also, you won't notice this immediately, because if you're still in the
do-this/fix-that phase, you won't learn a lot (nor will you teach a lot)).

I think the crux is that people must encounter it, and by limiting the
exercises, that was done, and I think it worked quite well.

Maybe we need to rethink that feature? Why was it introduced? What problem
does it solve, and can we solve it in some other way, while also solving
these problems?

I think it was introduced because people got bored, because they were
stuck, and had no way to move forward. Maybe we could unlock it after some
time (1 hour, 1 day, 1 week?) instead of immediately? A more forceful
alternative would be to unlock later exercises only after giving out
nitpicks (but that could have very negative consequences, though I suspect
they might be less dire here then on some random commenting site; it might
even work ;) ).

To come back to the other part of the problem, maybe we need to limit that
behaviour as well. Maybe you should only be allowed to do 20 nitpicks a
day? Or only nitpick half of the pending submissions per exercise?

Mark

Re: [exercism] On the lack of participants and the Zealous Teacher Paradox

From:
Peter Minten
Date:
2014-02-11 @ 07:47
Quick thought: mark submissions of people who have left a nit in the last 
few days so that we might look at their submissions first.

On 11 februari 2014 03:55:36 CET, Katrina Owen <_@kytrinyx.com> wrote:
>On 02/10, David Rueck wrote:
>> I agree that the ability to submit multiple exercises for the same
>language
>> at once could be contributing to people not nitpicking as much.
>
>I think you might be right, and at the same time, I don't yet want to
>switch it back.
>
>I'm hoping to find some sort of middle ground where people can work on
>several things at once, but where they maybe don't go hogwild.
>
>> It also definitely
>> contributes to a ton more work for the people who are committed to
>> nitpicking and makes it feel more one-sided, like there are
>"students" who
>> are submitting many many exercises and far fewer "teachers" who are
>trying
>> to wade through the giant list of multiple submissions per student.
>
>Yeah. There are "students" that I've stopped nitpicking, because it
>I get a bit grumpy that they've submitted 70+ exercises without ever
>nitpicking once.
>
>> This is sort of off topic, but related to the multiple submissions at
>once,
>> I've seen people make the same "mistake" in multiple exercises
>because they
>> submitted them all at once without first receiving feedback on the
>first
>> one.
>
>Agreed, that is why I originally had limited it.
>
>The thing that I don't seem to be able to convey to people is that
>they're likely to learn *more* from reading and dissecting other
>people's code, and trying to come up with a coherent way of expressing
>your thoughts about that code than by solving an exercise and moving
>on.
>
>Katrina

-- 
Verzonden van mijn Android telefoon met K-9 Mail.

Re: [exercism] On the lack of participants and the Zealous Teacher Paradox

From:
Matthew Marcus
Date:
2014-02-11 @ 13:15
Yeah, that last idea seems fair.  You don't have to enforce a rigid policy
of "you must provide a nitpick to receive a nitpick", to have some kind of
working karma system.

For the record, I click through to look at suggested submissions by other
people quite a lot, but don't always leave a nitpick.  As a newbie in most
of these languages myself,
it doesn't feel right to wade in with authoritative opinions (which is what
people seem to want I guess?).  If someone's done something that was
corrected on my own submission,
I always bring that up though.  I don't know - when confronted with a
submission that I don't fully comprehend, by someone who may be better with
the language than I am, is it a
valid nitpick to ask them what's going on in their code?  That seems like a
pretty unusual definition of the word "nitpick".


On Tue, Feb 11, 2014 at 7:47 AM, Peter Minten <peter@pminten.nl> wrote:

> Quick thought: mark submissions of people who have left a nit in the last
> few days so that we might look at their submissions first.
>
>
> On 11 februari 2014 03:55:36 CET, Katrina Owen <_@kytrinyx.com> wrote:
>>
>> On 02/10, David Rueck wrote:
>>
>>>  I agree that the ability to submit multiple exercises for the same language
>>>  at once could be contributing to people not nitpicking as much.
>>>
>>
>> I think you might be right, and at the same time, I don't yet want to
>> switch it back.
>>
>> I'm hoping to find some sort of middle ground where people can work on
>> several things at once, but where they maybe don't go hogwild.
>>
>>  It also definitely
>>>  contributes to a ton more work for the people who are committed to
>>>  nitpicking and makes it feel more one-sided, like there are "students" who
>>>  are submitting many many exercises and far fewer "teachers" who are trying
>>>  to wade through the gian
>>>  t list of multiple submissions per student.
>>>
>>
>> Yeah. There are "students" that I've stopped nitpicking, because it
>> I get a bit grumpy that they've submitted 70+ exercises without ever
>> nitpicking once.
>>
>>  This is sort of off topic, but related to the multiple submissions at once,
>>>  I've seen people make the same "mistake" in multiple exercises because they
>>>  submitted them all at once without first receiving feedback on the first
>>>  one.
>>>
>>
>> Agreed, that is why I originally had limited it.
>>
>> The thing that I don't seem to be able to convey to people is that
>> they're likely to learn *more* from reading and dissecting other
>> people's code, and trying to come up with a coherent way of expressing
>> your thoughts about that code than by solving an exercise and moving on.
>>
>> K
>>  atrina
>>
>>
> --
> Verzonden van mijn Android telefoon met K-9 Mail.
>

Re: [exercism] On the lack of participants and the Zealous Teacher Paradox

From:
Mark IJbema
Date:
2014-02-11 @ 13:29
I think if you don't comprehend a submission that can be very valuable
feedback. Imho you shouldn't refrain from asking why something was done in
a certain way. Maybe we should rename 'nitpick' though I really like how it
sounds sort of witty and non-serious.


On Tue, Feb 11, 2014 at 2:15 PM, Matthew Marcus <osirun@gmail.com> wrote:

> Yeah, that last idea seems fair.  You don't have to enforce a rigid policy
> of "you must provide a nitpick to receive a nitpick", to have some kind of
> working karma system.
>
> For the record, I click through to look at suggested submissions by other
> people quite a lot, but don't always leave a nitpick.  As a newbie in most
> of these languages myself,
> it doesn't feel right to wade in with authoritative opinions (which is
> what people seem to want I guess?).  If someone's done something that was
> corrected on my own submission,
> I always bring that up though.  I don't know - when confronted with a
> submission that I don't fully comprehend, by someone who may be better with
> the language than I am, is it a
> valid nitpick to ask them what's going on in their code?  That seems like
> a pretty unusual definition of the word "nitpick".
>
>
> On Tue, Feb 11, 2014 at 7:47 AM, Peter Minten <peter@pminten.nl> wrote:
>
>> Quick thought: mark submissions of people who have left a nit in the last
>> few days so that we might look at their submissions first.
>>
>>
>> On 11 februari 2014 03:55:36 CET, Katrina Owen <_@kytrinyx.com> wrote:
>>>
>>> On 02/10, David Rueck wrote:
>>>
>>>>  I agree that the ability to submit multiple exercises for the same language
>>>>
>>>>  at once could be contributing to people not nitpicking as much.
>>>>
>>>
>>> I think you might be right, and at the same time, I don't yet want to
>>> switch it back.
>>>
>>> I'm hoping to find some sort of middle ground where people can work on
>>>
>>> several things at once, but where they maybe don't go hogwild.
>>>
>>>  It also definitely
>>>>  contributes to a ton more work for the people who are committed to
>>>>
>>>>  nitpicking and makes it feel more one-sided, like there are "students" who
>>>>  are submitting many many exercises and far fewer "teachers" who are trying
>>>>  to wade through the gian
>>>>  t list of multiple submissions per student.
>>>>
>>>
>>> Yeah. There are "students" that I've stopped nitpicking, because it
>>> I get a bit grumpy that they've submitted 70+ exercises without ever
>>>
>>> nitpicking once.
>>>
>>>  This is sort of off topic, but related to the multiple submissions at once,
>>>>  I've seen people make the same "mistake" in multiple exercises because they
>>>>
>>>>  submitted them all at once without first receiving feedback on the first
>>>>  one.
>>>>
>>>
>>> Agreed, that is why I originally had limited it.
>>>
>>> The thing that I don't seem to be able to convey to people is that
>>>
>>> they're likely to learn *more* from reading and dissecting other
>>> people's code, and trying to come up with a coherent way of expressing
>>> your thoughts about that code than by solving an exercise and moving on.
>>>
>>> K
>>>  atrina
>>>
>>>
>> --
>> Verzonden van mijn Android telefoon met K-9 Mail.
>>
>
>

Re: [exercism] On the lack of participants and the Zealous Teacher Paradox

From:
Peter Minten
Date:
2014-02-11 @ 17:59
I'm beginning to believe that actually the most valuable feedback isn't
that of a master to a pupil but of one journeyman to another. Advice
from someone who you think if much more knowledgeable will be more
likely to be accepted without critical thought while the process of
critically looking at feedback that's being given is what you learn the
most from. Conversely you learn the most from teaching if you are
sometimes wrong and corrected by the student.

On 11/02/14 14:29, Mark IJbema wrote:
> I think if you don't comprehend a submission that can be very valuable
> feedback. Imho you shouldn't refrain from asking why something was done in
> a certain way. Maybe we should rename 'nitpick' though I really like how it
> sounds sort of witty and non-serious.
> 
> 
> On Tue, Feb 11, 2014 at 2:15 PM, Matthew Marcus <osirun@gmail.com> wrote:
> 
>> Yeah, that last idea seems fair.  You don't have to enforce a rigid policy
>> of "you must provide a nitpick to receive a nitpick", to have some kind of
>> working karma system.
>>
>> For the record, I click through to look at suggested submissions by other
>> people quite a lot, but don't always leave a nitpick.  As a newbie in most
>> of these languages myself,
>> it doesn't feel right to wade in with authoritative opinions (which is
>> what people seem to want I guess?).  If someone's done something that was
>> corrected on my own submission,
>> I always bring that up though.  I don't know - when confronted with a
>> submission that I don't fully comprehend, by someone who may be better with
>> the language than I am, is it a
>> valid nitpick to ask them what's going on in their code?  That seems like
>> a pretty unusual definition of the word "nitpick".
>>
>>
>> On Tue, Feb 11, 2014 at 7:47 AM, Peter Minten <peter@pminten.nl> wrote:
>>
>>> Quick thought: mark submissions of people who have left a nit in the last
>>> few days so that we might look at their submissions first.
>>>
>>>
>>> On 11 februari 2014 03:55:36 CET, Katrina Owen <_@kytrinyx.com> wrote:
>>>>
>>>> On 02/10, David Rueck wrote:
>>>>
>>>>>  I agree that the ability to submit multiple exercises for the same language
>>>>>
>>>>>  at once could be contributing to people not nitpicking as much.
>>>>>
>>>>
>>>> I think you might be right, and at the same time, I don't yet want to
>>>> switch it back.
>>>>
>>>> I'm hoping to find some sort of middle ground where people can work on
>>>>
>>>> several things at once, but where they maybe don't go hogwild.
>>>>
>>>>  It also definitely
>>>>>  contributes to a ton more work for the people who are committed to
>>>>>
>>>>>  nitpicking and makes it feel more one-sided, like there are "students" who
>>>>>  are submitting many many exercises and far fewer "teachers" who are trying
>>>>>  to wade through the gian
>>>>>  t list of multiple submissions per student.
>>>>>
>>>>
>>>> Yeah. There are "students" that I've stopped nitpicking, because it
>>>> I get a bit grumpy that they've submitted 70+ exercises without ever
>>>>
>>>> nitpicking once.
>>>>
>>>>  This is sort of off topic, but related to the multiple submissions at once,
>>>>>  I've seen people make the same "mistake" in multiple exercises because they
>>>>>
>>>>>  submitted them all at once without first receiving feedback on the first
>>>>>  one.
>>>>>
>>>>
>>>> Agreed, that is why I originally had limited it.
>>>>
>>>> The thing that I don't seem to be able to convey to people is that
>>>>
>>>> they're likely to learn *more* from reading and dissecting other
>>>> people's code, and trying to come up with a coherent way of expressing
>>>> your thoughts about that code than by solving an exercise and moving on.
>>>>
>>>> K
>>>>  atrina
>>>>
>>>>
>>> --
>>> Verzonden van mijn Android telefoon met K-9 Mail.
>>>
>>
>>
> 

Re: [exercism] On the lack of participants and the Zealous Teacher Paradox

From:
Katrina Owen
Date:
2014-02-11 @ 18:05
On 02/11, Peter Minten wrote:
> I'm beginning to believe that actually the most valuable feedback isn't
> that of a master to a pupil but of one journeyman to another.

I believe that this is true, but not for novices. Advice from a novice
to another novice can be pretty disastrous, leading both people down
completely unhelpful paths.

I've been trying to change the way I nitpick to ask open-ended questions
rather than say what I think all the time. Sometimes I really am
wondering about things, of course, but other times I just want to have
them think about something that I found interesting to think about.

It's very challenging (and I still pretty much suck at it).

Katrina

Re: [exercism] On the lack of participants and the Zealous Teacher Paradox

From:
Henrik Nyh
Date:
2014-02-11 @ 18:34
On Tue, Feb 11, 2014 at 7:05 PM, Katrina Owen <_@kytrinyx.com> wrote:

> I've been trying to change the way I nitpick to ask open-ended questions
> rather than say what I think all the time. Sometimes I really am
> wondering about things, of course, but other times I just want to have
> them think about something that I found interesting to think about.
>
> It's very challenging (and I still pretty much suck at it).
>

Very much agree (though not that you suck :p). We have actual meatspace
junior developers at work, and it's a big challenge. It's so easy to lead
them to paint your own solution by numbers, but it is probably a better
idea to ask questions to make them discover things on their own. Easier
said than done.

Re: [exercism] On the lack of participants and the Zealous Teacher Paradox

From:
Mike Coutermarsh
Date:
2014-02-11 @ 13:33
What if the user could only access and submit the first 5 exercises
initially?

Then once they nitpick a submission, the next 5 are opened up?

I think getting someone to do their first nitpick is the hurdle. Once you
do 1 and see that you learn from it... It becomes more likely they'll
participate.

I think implementing a system like this is something that's very easy to
get wrong. So keeping it super simple at first is important.

-Mike

(sent from phone)
On Feb 11, 2014 8:16 AM, "Matthew Marcus" <osirun@gmail.com> wrote:

> Yeah, that last idea seems fair.  You don't have to enforce a rigid policy
> of "you must provide a nitpick to receive a nitpick", to have some kind of
> working karma system.
>
> For the record, I click through to look at suggested submissions by other
> people quite a lot, but don't always leave a nitpick.  As a newbie in most
> of these languages myself,
> it doesn't feel right to wade in with authoritative opinions (which is
> what people seem to want I guess?).  If someone's done something that was
> corrected on my own submission,
> I always bring that up though.  I don't know - when confronted with a
> submission that I don't fully comprehend, by someone who may be better with
> the language than I am, is it a
> valid nitpick to ask them what's going on in their code?  That seems like
> a pretty unusual definition of the word "nitpick".
>
>
> On Tue, Feb 11, 2014 at 7:47 AM, Peter Minten <peter@pminten.nl> wrote:
>
>> Quick thought: mark submissions of people who have left a nit in the last
>> few days so that we might look at their submissions first.
>>
>>
>> On 11 februari 2014 03:55:36 CET, Katrina Owen <_@kytrinyx.com> wrote:
>>>
>>> On 02/10, David Rueck wrote:
>>>
>>>>  I agree that the ability to submit multiple exercises for the same language
>>>>
>>>>  at once could be contributing to people not nitpicking as much.
>>>>
>>>
>>> I think you might be right, and at the same time, I don't yet want to
>>> switch it back.
>>>
>>> I'm hoping to find some sort of middle ground where people can work on
>>>
>>> several things at once, but where they maybe don't go hogwild.
>>>
>>>  It also definitely
>>>>  contributes to a ton more work for the people who are committed to
>>>>
>>>>  nitpicking and makes it feel more one-sided, like there are "students" who
>>>>  are submitting many many exercises and far fewer "teachers" who are trying
>>>>  to wade through the gian
>>>>  t list of multiple submissions per student.
>>>>
>>>
>>> Yeah. There are "students" that I've stopped nitpicking, because it
>>> I get a bit grumpy that they've submitted 70+ exercises without ever
>>>
>>> nitpicking once.
>>>
>>>  This is sort of off topic, but related to the multiple submissions at once,
>>>>  I've seen people make the same "mistake" in multiple exercises because they
>>>>
>>>>  submitted them all at once without first receiving feedback on the first
>>>>  one.
>>>>
>>>
>>> Agreed, that is why I originally had limited it.
>>>
>>> The thing that I don't seem to be able to convey to people is that
>>>
>>> they're likely to learn *more* from reading and dissecting other
>>> people's code, and trying to come up with a coherent way of expressing
>>> your thoughts about that code than by solving an exercise and moving on.
>>>
>>> K
>>>  atrina
>>>
>>>
>> --
>> Verzonden van mijn Android telefoon met K-9 Mail.
>>
>
>

Re: [exercism] On the lack of participants and the Zealous Teacher Paradox

From:
Geoff Lanotte
Date:
2014-02-11 @ 13:40
I am new to Exercism, and have been using it fairly extensively that last
two weeks. While I feel comfortable nitpicking around in Ruby, I am
somewhat hesitant to nitpick other languages where I am less familiar with
best practices and language-specific idioms, particularly in Elixir. I do
spend quite a bit of time reading other people's nitpicks of pending
submissions in an effort to level up. I have seen an issue for this
already, but perhaps enabling people to review previous submissions that
are not pending and the related conversation could be helpful in directing
people.

One thought, perhaps offering nitpicking guides that are language and
exercise specific. What are the common issues, how best to help people
resolve them. Or even just maintain a list of good examples of nitpicking
specific exercises.

And for fear of making things way too gimicky, there is always gamification
to elicit specific desired bahaviors. Badges, titles, points, etc.



On Tue, Feb 11, 2014 at 8:15 AM, Matthew Marcus <osirun@gmail.com> wrote:

> Yeah, that last idea seems fair.  You don't have to enforce a rigid policy
> of "you must provide a nitpick to receive a nitpick", to have some kind of
> working karma system.
>
> For the record, I click through to look at suggested submissions by other
> people quite a lot, but don't always leave a nitpick.  As a newbie in most
> of these languages myself,
> it doesn't feel right to wade in with authoritative opinions (which is
> what people seem to want I guess?).  If someone's done something that was
> corrected on my own submission,
> I always bring that up though.  I don't know - when confronted with a
> submission that I don't fully comprehend, by someone who may be better with
> the language than I am, is it a
> valid nitpick to ask them what's going on in their code?  That seems like
> a pretty unusual definition of the word "nitpick".
>
>
> On Tue, Feb 11, 2014 at 7:47 AM, Peter Minten <peter@pminten.nl> wrote:
>
>> Quick thought: mark submissions of people who have left a nit in the last
>> few days so that we might look at their submissions first.
>>
>>
>> On 11 februari 2014 03:55:36 CET, Katrina Owen <_@kytrinyx.com> wrote:
>>>
>>> On 02/10, David Rueck wrote:
>>>
>>>>  I agree that the ability to submit multiple exercises for the same language
>>>>
>>>>  at once could be contributing to people not nitpicking as much.
>>>>
>>>
>>> I think you might be right, and at the same time, I don't yet want to
>>> switch it back.
>>>
>>> I'm hoping to find some sort of middle ground where people can work on
>>>
>>> several things at once, but where they maybe don't go hogwild.
>>>
>>>  It also definitely
>>>>  contributes to a ton more work for the people who are committed to
>>>>
>>>>  nitpicking and makes it feel more one-sided, like there are "students" who
>>>>  are submitting many many exercises and far fewer "teachers" who are trying
>>>>  to wade through the gian
>>>>  t list of multiple submissions per student.
>>>>
>>>
>>> Yeah. There are "students" that I've stopped nitpicking, because it
>>> I get a bit grumpy that they've submitted 70+ exercises without ever
>>>
>>> nitpicking once.
>>>
>>>  This is sort of off topic, but related to the multiple submissions at once,
>>>>  I've seen people make the same "mistake" in multiple exercises because they
>>>>
>>>>  submitted them all at once without first receiving feedback on the first
>>>>  one.
>>>>
>>>
>>> Agreed, that is why I originally had limited it.
>>>
>>> The thing that I don't seem to be able to convey to people is that
>>>
>>> they're likely to learn *more* from reading and dissecting other
>>> people's code, and trying to come up with a coherent way of expressing
>>> your thoughts about that code than by solving an exercise and moving on.
>>>
>>> K
>>>  atrina
>>>
>>>
>> --
>> Verzonden van mijn Android telefoon met K-9 Mail.
>>
>
>

Re: [exercism] On the lack of participants and the Zealous Teacher Paradox

From:
Michael Westbom
Date:
2014-02-11 @ 23:10
On Feb 11, 2014, at 8:40 AM, Geoff Lanotte <geofflanotte@gmail.com> wrote:
> I am somewhat hesitant to nitpick other languages where I am less 
familiar with best practices and language-specific idioms, particularly in
Elixir.

You are in the *perfect* position to nitpick then :)  If you see anything 
that rubs you as wanting change, maybe suggest your change but ask why the
user made the decision they did.  

The beauty of nitpicking, in that I understand it, is not that you are 
there to go over others’ code with a red pen, but to ask, suggest, and 
learn.  The idea is that the submitter pushes back if they believe in 
their code, providing an opportunity for both parties to learn.

If you don’t feel comfortable guiding, then ask. :)

Michael

Re: [exercism] On the lack of participants and the Zealous Teacher Paradox

From:
Matthew Marcus
Date:
2014-02-11 @ 13:48
Have to say, I think it's a bit weird that (in a fit of renewed nitpicking
purposefulness) I just clicked on "why not try nitpicking this?" link and
got a submission which 3 people have already agreed "looks great"...


On Tue, Feb 11, 2014 at 1:40 PM, Geoff Lanotte <geofflanotte@gmail.com>wrote:

> I am new to Exercism, and have been using it fairly extensively that last
> two weeks. While I feel comfortable nitpicking around in Ruby, I am
> somewhat hesitant to nitpick other languages where I am less familiar with
> best practices and language-specific idioms, particularly in Elixir. I do
> spend quite a bit of time reading other people's nitpicks of pending
> submissions in an effort to level up. I have seen an issue for this
> already, but perhaps enabling people to review previous submissions that
> are not pending and the related conversation could be helpful in directing
> people.
>
> One thought, perhaps offering nitpicking guides that are language and
> exercise specific. What are the common issues, how best to help people
> resolve them. Or even just maintain a list of good examples of nitpicking
> specific exercises.
>
> And for fear of making things way too gimicky, there is always
> gamification to elicit specific desired bahaviors. Badges, titles, points,
> etc.
>
>
>
> On Tue, Feb 11, 2014 at 8:15 AM, Matthew Marcus <osirun@gmail.com> wrote:
>
>> Yeah, that last idea seems fair.  You don't have to enforce a rigid
>> policy of "you must provide a nitpick to receive a nitpick", to have some
>> kind of working karma system.
>>
>> For the record, I click through to look at suggested submissions by other
>> people quite a lot, but don't always leave a nitpick.  As a newbie in most
>> of these languages myself,
>> it doesn't feel right to wade in with authoritative opinions (which is
>> what people seem to want I guess?).  If someone's done something that was
>> corrected on my own submission,
>> I always bring that up though.  I don't know - when confronted with a
>> submission that I don't fully comprehend, by someone who may be better with
>> the language than I am, is it a
>> valid nitpick to ask them what's going on in their code?  That seems like
>> a pretty unusual definition of the word "nitpick".
>>
>>
>> On Tue, Feb 11, 2014 at 7:47 AM, Peter Minten <peter@pminten.nl> wrote:
>>
>>> Quick thought: mark submissions of people who have left a nit in the
>>> last few days so that we might look at their submissions first.
>>>
>>>
>>> On 11 februari 2014 03:55:36 CET, Katrina Owen <_@kytrinyx.com> wrote:
>>>>
>>>> On 02/10, David Rueck wrote:
>>>>
>>>>>  I agree that the ability to submit multiple exercises for the same language
>>>>>
>>>>>
>>>>>  at once could be contributing to people not nitpicking as much.
>>>>>
>>>>
>>>> I think you might be right, and at the same time, I don't yet want to
>>>> switch it back.
>>>>
>>>> I'm hoping to find some sort of middle ground where people can work on
>>>>
>>>>
>>>> several things at once, but where they maybe don't go hogwild.
>>>>
>>>>  It also definitely
>>>>>  contributes to a ton more work for the people who are committed to
>>>>>
>>>>>
>>>>>  nitpicking and makes it feel more one-sided, like there are "students" who
>>>>>  are submitting many many exercises and far fewer "teachers" who are trying
>>>>>  to wade through the gian
>>>>>  t list of multiple submissions per student.
>>>>>
>>>>
>>>> Yeah. There are "students" that I've stopped nitpicking, because it
>>>> I get a bit grumpy that they've submitted 70+ exercises without ever
>>>>
>>>>
>>>> nitpicking once.
>>>>
>>>>  This is sort of off topic, but related to the multiple submissions at once,
>>>>>  I've seen people make the same "mistake" in multiple exercises because they
>>>>>
>>>>>
>>>>>  submitted them all at once without first receiving feedback on the first
>>>>>  one.
>>>>>
>>>>
>>>> Agreed, that is why I originally had limited it.
>>>>
>>>> The thing that I don't seem to be able to convey to people is that
>>>>
>>>>
>>>> they're likely to learn *more* from reading and dissecting other
>>>> people's code, and trying to come up with a coherent way of expressing
>>>> your thoughts about that code than by solving an exercise and moving on.
>>>>
>>>>
>>>> K
>>>>  atrina
>>>>
>>>>
>>> --
>>> Verzonden van mijn Android telefoon met K-9 Mail.
>>>
>>
>>
>

Re: [exercism] On the lack of participants and the Zealous Teacher Paradox

From:
Mike Coutermarsh
Date:
2014-02-11 @ 13:53
Sounds like that shouldn't happen. Maybe make a github issue. Then can be
discussed/fixed.

(sent from phone)
On Feb 11, 2014 8:50 AM, "Matthew Marcus" <osirun@gmail.com> wrote:

> Have to say, I think it's a bit weird that (in a fit of renewed nitpicking
> purposefulness) I just clicked on "why not try nitpicking this?" link and
> got a submission which 3 people have already agreed "looks great"...
>
>
> On Tue, Feb 11, 2014 at 1:40 PM, Geoff Lanotte <geofflanotte@gmail.com>wrote:
>
>> I am new to Exercism, and have been using it fairly extensively that last
>> two weeks. While I feel comfortable nitpicking around in Ruby, I am
>> somewhat hesitant to nitpick other languages where I am less familiar with
>> best practices and language-specific idioms, particularly in Elixir. I do
>> spend quite a bit of time reading other people's nitpicks of pending
>> submissions in an effort to level up. I have seen an issue for this
>> already, but perhaps enabling people to review previous submissions that
>> are not pending and the related conversation could be helpful in directing
>> people.
>>
>> One thought, perhaps offering nitpicking guides that are language and
>> exercise specific. What are the common issues, how best to help people
>> resolve them. Or even just maintain a list of good examples of nitpicking
>> specific exercises.
>>
>> And for fear of making things way too gimicky, there is always
>> gamification to elicit specific desired bahaviors. Badges, titles, points,
>> etc.
>>
>>
>>
>> On Tue, Feb 11, 2014 at 8:15 AM, Matthew Marcus <osirun@gmail.com> wrote:
>>
>>> Yeah, that last idea seems fair.  You don't have to enforce a rigid
>>> policy of "you must provide a nitpick to receive a nitpick", to have some
>>> kind of working karma system.
>>>
>>> For the record, I click through to look at suggested submissions by
>>> other people quite a lot, but don't always leave a nitpick.  As a newbie in
>>> most of these languages myself,
>>> it doesn't feel right to wade in with authoritative opinions (which is
>>> what people seem to want I guess?).  If someone's done something that was
>>> corrected on my own submission,
>>> I always bring that up though.  I don't know - when confronted with a
>>> submission that I don't fully comprehend, by someone who may be better with
>>> the language than I am, is it a
>>> valid nitpick to ask them what's going on in their code?  That seems
>>> like a pretty unusual definition of the word "nitpick".
>>>
>>>
>>> On Tue, Feb 11, 2014 at 7:47 AM, Peter Minten <peter@pminten.nl> wrote:
>>>
>>>> Quick thought: mark submissions of people who have left a nit in the
>>>> last few days so that we might look at their submissions first.
>>>>
>>>>
>>>> On 11 februari 2014 03:55:36 CET, Katrina Owen <_@kytrinyx.com> wrote:
>>>>>
>>>>> On 02/10, David Rueck wrote:
>>>>>
>>>>>>  I agree that the ability to submit multiple exercises for the same
language
>>>>>>
>>>>>>
>>>>>>
>>>>>>  at once could be contributing to people not nitpicking as much.
>>>>>>
>>>>>
>>>>> I think you might be right, and at the same time, I don't yet want to
>>>>> switch it back.
>>>>>
>>>>> I'm hoping to find some sort of middle ground where people can work on
>>>>>
>>>>>
>>>>>
>>>>> several things at once, but where they maybe don't go hogwild.
>>>>>
>>>>>  It also definitely
>>>>>>  contributes to a ton more work for the people who are committed to
>>>>>>
>>>>>>
>>>>>>
>>>>>>  nitpicking and makes it feel more one-sided, like there are "students" who
>>>>>>  are submitting many many exercises and far fewer "teachers" who are trying
>>>>>>  to wade through the gian
>>>>>>  t list of multiple submissions per student.
>>>>>>
>>>>>
>>>>> Yeah. There are "students" that I've stopped nitpicking, because it
>>>>> I get a bit grumpy that they've submitted 70+ exercises without ever
>>>>>
>>>>>
>>>>>
>>>>> nitpicking once.
>>>>>
>>>>>  This is sort of off topic, but related to the multiple submissions at once,
>>>>>>  I've seen people make the same "mistake" in multiple exercises 
because they
>>>>>>
>>>>>>
>>>>>>
>>>>>>  submitted them all at once without first receiving feedback on the first
>>>>>>  one.
>>>>>>
>>>>>
>>>>> Agreed, that is why I originally had limited it.
>>>>>
>>>>> The thing that I don't seem to be able to convey to people is that
>>>>>
>>>>>
>>>>>
>>>>> they're likely to learn *more* from reading and dissecting other
>>>>> people's code, and trying to come up with a coherent way of expressing
>>>>> your thoughts about that code than by solving an exercise and moving on.
>>>>>
>>>>>
>>>>>
>>>>> K
>>>>>  atrina
>>>>>
>>>>>
>>>> --
>>>> Verzonden van mijn Android telefoon met K-9 Mail.
>>>>
>>>
>>>
>>
>

Re: [exercism] On the lack of participants and the Zealous Teacher Paradox

From:
Katrina Owen
Date:
2014-02-11 @ 02:49
On 02/10, Peter Minten wrote:
> Let me first define a few terms:
>
> "student": Someone who sends in submissions but does not nitpick.
> "teacher": Someone who nitpicks but does not send in submissions.
> "participant: Someone who sends in submissions and also nitpicks.
>
> I've noticed a paradox with regard to teachers, something I call the
> Zealous Teacher Paradox: if you have a teacher who is knowledgeable and
> who responds to every submission or comment within a reasonable time
> frame the track will suffer as a result.

This matches my experience as well.

What I really want is lots and lots of participants.

> Unfortunately I get the impression that the student->participant
> conversion ratio has become worse lately. I suspect it's related to the
> change that you can have multiple submissions pending at the same time.

I think you might be right, but I haven't made any measurements in this
regard.

I have about a dozen things I want to do to improve the site, and
life/dayjob/etc has gotten in the way, and this hasn't floated to the
top yet.

> The conversion of students to participants on the other hand seems to be
> fuelled by a sense of familiarity with the site and the nitpickers on a
> track. The best participants I've seen are those that I've had
> conversations with over a long series of submissions.

Agreed, this is something that I've noticed in my own interactions with
people as well.

> There are no obvious solutions to converting more students into
> participants, but it's important that we think about this because in the
> end exercism needs to encourage enough people to become participants in
> order to grow.

Thank you very much for thinking out loud here. These are incredibly
important issues, and I would love to have help figuring out how to
tackle them.

Katrina