librelist archives

Re: lamson-1.0pre4

From:

Stéphane Fornengo

Date:

2009-09-08 @ 15:43

Hi,
I do not always succeed in testing lamson.confirm... here is my log file:

2009-09-08 17:37:15,320 - root - INFO - SMTPReceiver started on
localhost:8823.
2009-09-08 17:37:57,389 - root - DEBUG - Message received from Peer:
('127.0.0.1', 57932), From: 'stf@naos', to To ['post@mysite.com'].
2009-09-08 17:37:57,404 - routing - DEBUG - Matched 'post@mysite.com'
against START.
2009-09-08 17:37:57,661 - routing - DEBUG - Message to set(['post@mysite.com'])
was handled by app.handlers.post.START
2009-09-08 17:38:27,664 - root - DEBUG - Message received from Peer:
('127.0.0.1', 57935), From: 'stf@naos', to To ['
post-confirm-44feb30819114ed5a3da1700b6f90d02@mysite.com'].
2009-09-08 17:38:27,668 - routing - DEBUG - Matched '
post-confirm-44feb30819114ed5a3da1700b6f90d02@mysite.com' against
CONFIRMING.
2009-09-08 17:38:27,695 - root - WARNING - Invalid confirm from stf
<stf@naos>
2009-09-08 17:38:27,695 - routing - DEBUG - Message to set(['
post-confirm-44feb30819114ed5a3da1700b6f90d02@mysite.com']) was handled by
app.handlers.post.CONFIRMING

as you see, there is a invalid confirm although the target, the from_address
and the expected secret are good in my bdd (I am using django storage).
Can you suggest me some help ?



2009/9/6 Stéphane FORNENGO <sfornengo@gmail.com>

>
> Hi,
>> I try lamson.confirm but I encounter some problems.
>> First, with shelf storage I get this error:
>> 2009-09-06 11:02:30,347 - routing - ERROR - !!! ERROR handling
>> app.handlers.post.START
>> Traceback (most recent call last):
>>   File
>> 
"/usr/local/lib/python2.6/dist-packages/lamson-1.0pre4-py2.6.egg/lamson/routing.py",
>> line 371, in call_safely
>>     func(message, **kwargs)
>>   File
>> 
"/usr/local/lib/python2.6/dist-packages/lamson-1.0pre4-py2.6.egg/lamson/routing.py",
>> line 490, in routing_wrapper
>>     next_state = func(message, *args, **kw)
>>   File
>> 
"/home/stf/backup_naos/proj/web/mailbazar/lamsonmailserver/app/handlers/post.py",
>> line 8, in START
>>     CONFIRM.send(relay, "post", message, "post/confirm.msg", locals())
>>   File
>> 
"/usr/local/lib/python2.6/dist-packages/lamson-1.0pre4-py2.6.egg/lamson/confirm.py",
>> line 179, in send
>>     confirm_address = self.register(target, message)
>>   File
>> 
"/usr/local/lib/python2.6/dist-packages/lamson-1.0pre4-py2.6.egg/lamson/confirm.py",
>> line 142, in register
>>     self.storage.store(target, from_address, secret, pending_id)
>>   File
>> 
"/usr/local/lib/python2.6/dist-packages/lamson-1.0pre4-py2.6.egg/lamson/confirm.py",
>> line 74, in store
>>     pending_message_id)
>>   File "/usr/lib/python2.6/shelve.py", line 133, in __setitem__
>>     self.dict[key] = f.getvalue()
>>   File "/usr/lib/python2.6/bsddb/__init__.py", line 276, in __setitem__
>>     _DeadlockWrap(wrapF)  # self.db[key] = value
>>   File "/usr/lib/python2.6/bsddb/dbutils.py", line 68, in DeadlockWrap
>>     return function(*_args, **_kwargs)
>>   File "/usr/lib/python2.6/bsddb/__init__.py", line 275, in wrapF
>>     self.db[key] = value
>> TypeError: String or Integer object expected for key, unicode found
>>
>> So I try to use default, a simple in-memory dict to store the
>> confirmations. No more error. But using mutt to confim the post, with the
>> good target, reply address and secret never succed in,
>> CONFIRM.verify('post', message['from'], id_number) return None.
>> Any idea ?
>> Thanks
>>
>
>

On Tue, Sep 08, 2009 at 05:43:24PM +0200, Stéphane FORNENGO wrote:
> Hi,
> I do not always succeed in testing lamson.confirm... here is my log file:
> 
> 
> as you see, there is a invalid confirm although the target, the from_address
> and the expected secret are good in my bdd (I am using django storage).
> Can you suggest me some help ?

Well, not sure without looking at your database, but it seems like you
might be storing the data wrong.  Did you write your own confirmation
storage or use the example?  Have you written a test that makes sure the
confirmation storage works?  Did you configure it with the confirmation
engine correctly?

I think if you look at how/where it's being stored you'll find it.

-- 
Zed A. Shaw
http://zedshaw.com/

I am using the example confirmation storage but I found the problem.
The data are correct in database but the problem are in the get method. the
from address to find (and set by mutt) is 'stf <stf@naos>' and the
from_address in my database is just 'stf@naos'. may be lamson need to clean
from_address before handle it ?

2009/9/8 Zed A. Shaw <zedshaw@zedshaw.com>

> On Tue, Sep 08, 2009 at 05:43:24PM +0200, Stéphane FORNENGO wrote:
> > Hi,
> > I do not always succeed in testing lamson.confirm... here is my log file:
> >
> >
> > as you see, there is a invalid confirm although the target, the
> from_address
> > and the expected secret are good in my bdd (I am using django storage).
> > Can you suggest me some help ?
>
> Well, not sure without looking at your database, but it seems like you
> might be storing the data wrong.  Did you write your own confirmation
> storage or use the example?  Have you written a test that makes sure the
> confirmation storage works?  Did you configure it with the confirmation
> engine correctly?
>
> I think if you look at how/where it's being stored you'll find it.
>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

On Wed, Sep 09, 2009 at 12:17:36PM +0200, Stéphane FORNENGO wrote:
> I am using the example confirmation storage but I found the problem.
> The data are correct in database but the problem are in the get method. the
> from address to find (and set by mutt) is 'stf <stf@naos>' and the
> from_address in my database is just 'stf@naos'. may be lamson need to clean
> from_address before handle it ?

Ah, yes, that's one more place I needed to convert to using the new
route_to, route_from attributes.  Looking at the API it'd be a big
change to use those directly, so I threw in a bit of code to cleanup the
address before putting it in storage where it's needed.

I checked it into bzr, or you can get the changes here:

easy_install http://lamsonproject.org/releases/lamson-1.0pre6.tar.gz

Try it and let me know if it works.

-- 
Zed A. Shaw
http://zedshaw.com/

ok.
another feature would be nice: allowing to define subject of
lamson.confim.send method. (I am a french man (this explain my poor english)
and the static text "confirmation required" isn't very appropriate. May be
subject can be a method parameter with a default text ?


2009/9/9 Zed A. Shaw <zedshaw@zedshaw.com>

> On Wed, Sep 09, 2009 at 12:17:36PM +0200, Stéphane FORNENGO wrote:
> > I am using the example confirmation storage but I found the problem.
> > The data are correct in database but the problem are in the get method.
> the
> > from address to find (and set by mutt) is 'stf <stf@naos>' and the
> > from_address in my database is just 'stf@naos'. may be lamson need to
> clean
> > from_address before handle it ?
>
> Ah, yes, that's one more place I needed to convert to using the new
> route_to, route_from attributes.  Looking at the API it'd be a big
> change to use those directly, so I threw in a bit of code to cleanup the
> address before putting it in storage where it's needed.
>
> I checked it into bzr, or you can get the changes here:
>
> easy_install http://lamsonproject.org/releases/lamson-1.0pre6.tar.gz
>
> Try it and let me know if it works.
>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

On Wed, Sep 09, 2009 at 08:15:58PM +0200, Stéphane FORNENGO wrote:
> ok.
> another feature would be nice: allowing to define subject of
> lamson.confim.send method. (I am a french man (this explain my poor english)
> and the static text "confirmation required" isn't very appropriate. May be
> subject can be a method parameter with a default text ?

Yeah sure, I'll put that in.  Can you think of other stuff?

-- 
Zed A. Shaw
http://zedshaw.com/

I am thinking about how to handle many confirmations at the same time. For
example just imagine an user who post many articles, so he receives many
confirmations like:
post-confirm-(id_number)@(host)
and now wants to confirm each article, one by one.
It seems that we get in trouble for doing that with lamson.confirm because
the get method of the storage class doesn't know about id_number.
Trying to solve this problem with different targets in order to have
different confirmation like:
post0-confirm-(id_number)@(host)
post1-confirm-(id_number)@(host)
post2-confirm-(id_number)@(host)
...
isn't efficient because it needs to hit database in order to know which
confimation is already stored for each new article...
so why using target in confirmation process ? why not just use id_number to
differentiate the confirmations and just use (from_addr, id_number) instead
of (from_addr, target) in the get method ?


2009/9/9 Zed A. Shaw <zedshaw@zedshaw.com>

> On Wed, Sep 09, 2009 at 08:15:58PM +0200, Stéphane FORNENGO wrote:
> > ok.
> > another feature would be nice: allowing to define subject of
> > lamson.confim.send method. (I am a french man (this explain my poor
> english)
> > and the static text "confirmation required" isn't very appropriate. May
> be
> > subject can be a method parameter with a default text ?
>
> Yeah sure, I'll put that in.  Can you think of other stuff?
>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

On Wed, Sep 09, 2009 at 09:50:29PM +0200, Stéphane FORNENGO wrote:
> I am thinking about how to handle many confirmations at the same time. For
> example just imagine an user who post many articles, so he receives many
> confirmations like:

The confirmation API is intended for confirming a at key states like
START, or after a bounce.  It isn't designed to confirm in complicated
situations like what you describe here.  In fact, it's fairly impossible
to write a confirmation API that would handle all the various ways
people would want to confirm users.

So, what you're supposed to do in this situation is:

1) Confirm that you are actually using it right, and that you really do
need something new.
2) If you do need something different, then write it.

So, if you really do need to do confirmations like this, then just use
the Lamson code as a template and write your own.  Chances are it won't
generalize to other people very well, so there's no need to include it
in the main source.

Make sense?

-- 
Zed A. Shaw
http://zedshaw.com/

yes, maybe my explanations are not very clear... here an other example:
an user send me a first article, and so hit my "START" state. my app (using
confimation API) send him a "confirmation requested" and store
(target,from_addr)=secret_id, waiting confirmation to go to the next state.
then my user does not confirm immediately his first article, he wants to
give some time before validating.
He sends me a second article, that hit my "START" state too. the
confirmation API, and it's the problem, store (target,from_addr)=secret_id
again, loosing the confirmation for the first article.
now trying to confirm his firt article can no longer work.

My first idea to solve this problem was to use confimation API (because its
goal is state transition, what I need) and just rewrite the confirmStorage,
using django. Taking the DjangoConfirmStorage example, we can see that the
store method store all state in database (ovoiding overwriting them unlike
dict storage above), but in the get method and only the (target,
from_address) parameters we are unable to select the good record. In your
DjangoConfirmStorage example you take the first from the list
(confirmations[0]) and it's not necessarily the good.
a simple solution: if the get method could have the expected_secret_id in
parameter, we could iterate on this list to find the good record, or
directly search in database for (target, from_address, expected_secret_id).

Make sense ?


2009/9/10 Zed A. Shaw <zedshaw@zedshaw.com>

> On Wed, Sep 09, 2009 at 09:50:29PM +0200, Stéphane FORNENGO wrote:
> > I am thinking about how to handle many confirmations at the same time.
> For
> > example just imagine an user who post many articles, so he receives many
> > confirmations like:
>
> The confirmation API is intended for confirming a at key states like
> START, or after a bounce.  It isn't designed to confirm in complicated
> situations like what you describe here.  In fact, it's fairly impossible
> to write a confirmation API that would handle all the various ways
> people would want to confirm users.
>
> So, what you're supposed to do in this situation is:
>
> 1) Confirm that you are actually using it right, and that you really do
> need something new.
> 2) If you do need something different, then write it.
>
> So, if you really do need to do confirmations like this, then just use
> the Lamson code as a template and write your own.  Chances are it won't
> generalize to other people very well, so there's no need to include it
> in the main source.
>
> Make sense?
>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

On Thu, Sep 10, 2009 at 09:58:49AM +0200, Stéphane FORNENGO wrote:
> yes, maybe my explanations are not very clear... here an other example:

Send code, that's always clear.

> an user send me a first article, and so hit my "START" state. my app (using
> confimation API) send him a "confirmation requested" and store
> (target,from_addr)=secret_id, waiting confirmation to go to the next state.
> then my user does not confirm immediately his first article, he wants to
> give some time before validating.

Nope, see, you're not using it right.  You send the confirmation, then
*change* to a CONFIRMING state, then the only thing they can do is
confirm.  That's the purpose of the state changes, so that you can
constrain the interactions so that you don't have these problems.


> He sends me a second article, that hit my "START" state too. the
> confirmation API, and it's the problem, store (target,from_addr)=secret_id
> again, loosing the confirmation for the first article.
> now trying to confirm his firt article can no longer work.

Yes, this is on purpose, store the confirmation, change to CONFIRMING
state, wait for confirmation.  You don't have this problem.  That's how
it works.
 
> My first idea to solve this problem was to use confimation API (because its
> goal is state transition, what I need) and just rewrite the confirmStorage,
> using django.

Yep, I figured this is what you were doing, which is why I said make
sure you're using it right.  I *REALLY* recommend you write some code to
do confirmations, compare it to the librelist code, and then if your
code is different, seriously examine why.  If you think it needs to be
different, then you'll have a code example to use that explains what you
want to do.

For example, I think you are doing this:

def START(...):
	if it's a confirmation
		verify confirmation
	else
		store confirmation
		send confiramation
	return START

When you *should* be doing this:

def START(...):
	store confirmation
	send confirmation
	return CONFIRMING  ## <-- KEY POINT, NOTICE!

def CONFIRMING(...):
	verify confirmation
	if valid
		return REGISTERED
	else
		return FAILED

See the difference?

-- 
Zed A. Shaw
http://zedshaw.com/

No, I am using the right structure:
def START(...):
       store confirmation
       send confirmation
       return CONFIRMING  ## <-- KEY POINT, NOTICE!

def CONFIRMING(...):
       verify confirmation
       if valid
               return REGISTERED
       else
               return FAILED

maybe I do not understand something...
I am going to write my own code to do confirmation to better understand,
thanks.




2009/9/10 Zed A. Shaw <zedshaw@zedshaw.com>

> On Thu, Sep 10, 2009 at 09:58:49AM +0200, Stéphane FORNENGO wrote:
> > yes, maybe my explanations are not very clear... here an other example:
>
> Send code, that's always clear.
>
> > an user send me a first article, and so hit my "START" state. my app
> (using
> > confimation API) send him a "confirmation requested" and store
> > (target,from_addr)=secret_id, waiting confirmation to go to the next
> state.
> > then my user does not confirm immediately his first article, he wants to
> > give some time before validating.
>
> Nope, see, you're not using it right.  You send the confirmation, then
> *change* to a CONFIRMING state, then the only thing they can do is
> confirm.  That's the purpose of the state changes, so that you can
> constrain the interactions so that you don't have these problems.
>
>
> > He sends me a second article, that hit my "START" state too. the
> > confirmation API, and it's the problem, store
> (target,from_addr)=secret_id
> > again, loosing the confirmation for the first article.
> > now trying to confirm his firt article can no longer work.
>
> Yes, this is on purpose, store the confirmation, change to CONFIRMING
> state, wait for confirmation.  You don't have this problem.  That's how
> it works.
>
> > My first idea to solve this problem was to use confimation API (because
> its
> > goal is state transition, what I need) and just rewrite the
> confirmStorage,
> > using django.
>
> Yep, I figured this is what you were doing, which is why I said make
> sure you're using it right.  I *REALLY* recommend you write some code to
> do confirmations, compare it to the librelist code, and then if your
> code is different, seriously examine why.  If you think it needs to be
> different, then you'll have a code example to use that explains what you
> want to do.
>
> For example, I think you are doing this:
>
> def START(...):
>        if it's a confirmation
>                verify confirmation
>        else
>                store confirmation
>                send confiramation
>        return START
>
> When you *should* be doing this:
>
> def START(...):
>        store confirmation
>        send confirmation
>        return CONFIRMING  ## <-- KEY POINT, NOTICE!
>
> def CONFIRMING(...):
>        verify confirmation
>        if valid
>                return REGISTERED
>        else
>                return FAILED
>
> See the difference?
>
> --
> Zed A. Shaw
> http://zedshaw.com/
>

On Thu, Sep 10, 2009 at 02:12:51PM +0200, Stéphane FORNENGO wrote:
> No, I am using the right structure:
> def START(...):
>        store confirmation
>        send confirmation
>        return CONFIRMING  ## <-- KEY POINT, NOTICE!
> 
> def CONFIRMING(...):
>        verify confirmation
>        if valid
>                return REGISTERED
>        else
>                return FAILED
> 
> maybe I do not understand something...
> I am going to write my own code to do confirmation to better understand,
> thanks.

Yeah, write some code that demonstrates the flaw you talk about,
with unit tests.  If you can write a unit test that does what you think
a user will do, and then write a "flawed" controller, then I can
understand what you mean.

-- 
Zed A. Shaw
http://zedshaw.com/