Re: [flask] Flask Application with multiprocessing and uwsgi
- From:
- Trey Long
- Date:
- 2013-01-28 @ 16:34
My advice would to be to use a separate process with communication between
the two or a task management system like Celery to do this for you.
http://celeryproject.org/
It seems like a less than stellar idea to try to attach this process into
your web stack.
On Jan 28, 2013, at 11:13 AM, "Olav Grønås Gjerde" <olavgg@gmail.com> wrote:
> I've developed a flask backend application that is syncing files and
> folders to an elasticsearch database.
>
> I've implemented the following logic for each request:
> First check if a process with the following name already exists in
> multiprocessing.Manager().dict()
> If not:
> create a new task which is added to multiproccesing.JoinableQueue
> Then create a worker which is an instance of -> class Worker(Process)
> And finally start the process that check if there are any jobs in
> queue and execute them one by one.
>
> Full implementation is available here:
> https://github.com/olavgg/py-sth/blob/master/client/services/task_service.py
>
> This works fine in development mode. But if I try to start this with
> uwsgi, the process will hang when executing the JoinableQueue.get()
> method (line 95)
>
> Am I doing this completely wrong? Are there any workarounds to handle
> expensive functions?