Processing incoming emails - Queuing them?
- From:
- Marc Eggenberger
- Date:
- 2011-12-31 @ 14:29
Hi there.
My name's Marc and I stumbled upon Lamson while researching for a project.
The original plan is to save/archive email messages into a store (NoSQL
database). This should be done in a distributed context. It's for a
research/educational project.
The original idea is to have postfix receive the mails, send them to a
server/service on a specifc port with the after-queue context filter
functionality of postfix. The service then takes the SMTP stream and sends
it to a APMQ. There will be worker process that pick up the work from the
APMQ and save it into the NoSQL store. The rational behind the after-queue
filter functionality is that as long as it's not in the APMQ and
acknowledged to the Postfix server (in case of an error/server down etc)
the sending mailserver will try it again as it wasn't able to handoff the
message (postfix will not acknowledge if he doesn't get the ok from the
after-queue filter). Much like an anti-spam setup.
I then found Lamson Project and was quite intrigued. Python. Nice
abstraction etc. So I'm looking into using Lamson as the primary receiver
of the emails. I saw the message queuing. So I would have one (or several?)
Lamson process receiving emails and queuing them. Another Lamson then
dequeues from it and processes it. Would that about be right?
Or use the maildir class from Python to access the queue? Would that be the
simpler solution? Can it handle multiple workers accessing the same maildir?
Thanks for any thoughts.
Regards,
Marc