librelist archives

« back to archive

Flask-SQLAlchemy + Celery

Flask-SQLAlchemy + Celery

From:
Panayiotis Papadopoulos
Date:
2012-08-20 @ 20:42
I recently switch to Celery 3.0. Before that I was using
Flask-Celery<https://github.com/ask/flask-celery/> in
order to integrate Celery with Flask. Although it had many issues like
hiding some powerful Celery functionalities but it allowed me to use the
full context of Flask app and especially Flask-SQLAlchemy.

In my background tasks I am processing data and the SQLAlchemy ORM to store
the data. The maintainer of Flask-Celery has dropped support of the plugin.
The plugin was pickling the Flask instance in the task so I could have full
access to SQLAlchemy.

I am trying to replicate this behavior in my tasks.py file but with no
success. Do you have any hints on how to achieve this?

-- 
Panos Papadopoulos
Founder and VP of Product at BugSense <http://www.bugsense.com/>
Follow us on Twitter @bugsense <http://twitter.com/bugsense>
US Cell (650) 267 0550  EU Cell +30 697 00 1 66 34

Re: [flask] Flask-SQLAlchemy + Celery

From:
Jonathan Zempel
Date:
2012-08-20 @ 20:56
I brought this up a few weeks ago. This reminds me I'm still hoping for a
reply (from Ask?). Here's the archived conversation link:


http://flask.pocoo.org/mailinglist/archive/2012/7/9/flask-celery-deprecated/#ffd4548f110c929c0bc3324154afdc5b

It contains gists that outline the problem you're seeing along with how I'd
like to see it handled. In short, I think we still want a working
Flask-Celery.

On Mon, Aug 20, 2012 at 1:42 PM, Panayiotis Papadopoulos <panosjee@gmail.com
> wrote:

> I recently switch to Celery 3.0. Before that I was using 
Flask-Celery<https://github.com/ask/flask-celery/> in
> order to integrate Celery with Flask. Although it had many issues like
> hiding some powerful Celery functionalities but it allowed me to use the
> full context of Flask app and especially Flask-SQLAlchemy.
>
> In my background tasks I am processing data and the SQLAlchemy ORM to
> store the data. The maintainer of Flask-Celery has dropped support of the
> plugin. The plugin was pickling the Flask instance in the task so I could
> have full access to SQLAlchemy.
>
> I am trying to replicate this behavior in my tasks.py file but with no
> success. Do you have any hints on how to achieve this?
>
> --
> Panos Papadopoulos
> Founder and VP of Product at BugSense <http://www.bugsense.com/>
> Follow us on Twitter @bugsense <http://twitter.com/bugsense>
> US Cell (650) 267 0550  EU Cell +30 697 00 1 66 34
>
>

Re: [flask] Flask-SQLAlchemy + Celery

From:
Panayiotis Papadopoulos
Date:
2012-08-21 @ 08:34
Thanks Jonathan, I am aware of this. I had found the Gist. It seems
Flask-Celery was pickling the whole flask app and this way recreating the
app context. I ll try to put up a similar solution (a bit dirty, isn't it)
and let you know

On Mon, Aug 20, 2012 at 11:56 PM, Jonathan Zempel <jzempel@gmail.com> wrote:

> I brought this up a few weeks ago. This reminds me I'm still hoping for a
> reply (from Ask?). Here's the archived conversation link:
>
>
> 
http://flask.pocoo.org/mailinglist/archive/2012/7/9/flask-celery-deprecated/#ffd4548f110c929c0bc3324154afdc5b
>
> It contains gists that outline the problem you're seeing along with how
> I'd like to see it handled. In short, I think we still want a working
> Flask-Celery.
>
>
> On Mon, Aug 20, 2012 at 1:42 PM, Panayiotis Papadopoulos <
> panosjee@gmail.com> wrote:
>
>> I recently switch to Celery 3.0. Before that I was using 
Flask-Celery<https://github.com/ask/flask-celery/> in
>> order to integrate Celery with Flask. Although it had many issues like
>> hiding some powerful Celery functionalities but it allowed me to use the
>> full context of Flask app and especially Flask-SQLAlchemy.
>>
>> In my background tasks I am processing data and the SQLAlchemy ORM to
>> store the data. The maintainer of Flask-Celery has dropped support of the
>> plugin. The plugin was pickling the Flask instance in the task so I could
>> have full access to SQLAlchemy.
>>
>> I am trying to replicate this behavior in my tasks.py file but with no
>> success. Do you have any hints on how to achieve this?
>>
>> --
>> Panos Papadopoulos
>> Founder and VP of Product at BugSense <http://www.bugsense.com/>
>> Follow us on Twitter @bugsense <http://twitter.com/bugsense>
>> US Cell (650) 267 0550  EU Cell +30 697 00 1 66 34
>>
>>
>


-- 
Panos Papadopoulos
Founder and VP of Product at BugSense <http://www.bugsense.com/>
Follow us on Twitter @bugsense <http://twitter.com/bugsense>
US Cell (650) 267 0550  EU Cell +30 697 00 1 66 34

Re: [flask] Flask-SQLAlchemy + Celery

From:
Panayiotis Papadopoulos
Date:
2012-08-21 @ 12:45
One simple solution is to use the test_request_context
Check out a small gist here:

http://stackoverflow.com/questions/12044776/how-to-use-flask-sqlalchemy-in-a-celery-task/12055084#12055084

On Tue, Aug 21, 2012 at 11:34 AM, Panayiotis Papadopoulos <
panosjee@gmail.com> wrote:

> Thanks Jonathan, I am aware of this. I had found the Gist. It seems
> Flask-Celery was pickling the whole flask app and this way recreating the
> app context. I ll try to put up a similar solution (a bit dirty, isn't it)
> and let you know
>
>
> On Mon, Aug 20, 2012 at 11:56 PM, Jonathan Zempel <jzempel@gmail.com>wrote:
>
>> I brought this up a few weeks ago. This reminds me I'm still hoping for a
>> reply (from Ask?). Here's the archived conversation link:
>>
>>
>> 
http://flask.pocoo.org/mailinglist/archive/2012/7/9/flask-celery-deprecated/#ffd4548f110c929c0bc3324154afdc5b
>>
>> It contains gists that outline the problem you're seeing along with how
>> I'd like to see it handled. In short, I think we still want a working
>> Flask-Celery.
>>
>>
>> On Mon, Aug 20, 2012 at 1:42 PM, Panayiotis Papadopoulos <
>> panosjee@gmail.com> wrote:
>>
>>> I recently switch to Celery 3.0. Before that I was using 
Flask-Celery<https://github.com/ask/flask-celery/> in
>>> order to integrate Celery with Flask. Although it had many issues like
>>> hiding some powerful Celery functionalities but it allowed me to use the
>>> full context of Flask app and especially Flask-SQLAlchemy.
>>>
>>> In my background tasks I am processing data and the SQLAlchemy ORM to
>>> store the data. The maintainer of Flask-Celery has dropped support of the
>>> plugin. The plugin was pickling the Flask instance in the task so I could
>>> have full access to SQLAlchemy.
>>>
>>> I am trying to replicate this behavior in my tasks.py file but with no
>>> success. Do you have any hints on how to achieve this?
>>>
>>> --
>>> Panos Papadopoulos
>>> Founder and VP of Product at BugSense <http://www.bugsense.com/>
>>> Follow us on Twitter @bugsense <http://twitter.com/bugsense>
>>> US Cell (650) 267 0550  EU Cell +30 697 00 1 66 34
>>>
>>>
>>
>
>
> --
> Panos Papadopoulos
> Founder and VP of Product at BugSense <http://www.bugsense.com/>
> Follow us on Twitter @bugsense <http://twitter.com/bugsense>
> US Cell (650) 267 0550  EU Cell +30 697 00 1 66 34
>
>


-- 
Panos Papadopoulos
Founder and VP of Product at BugSense <http://www.bugsense.com/>
Follow us on Twitter @bugsense <http://twitter.com/bugsense>
US Cell (650) 267 0550  EU Cell +30 697 00 1 66 34