Re: [flask] flask-celery deprecated?
- Jonathan Zempel
- 2012-07-29 @ 21:22
Thanks for getting back to me on this. I think this makes more sense if I
describe a scenario and then produce the issue with code.
The scenario is a large Flask application that sends email notifications
based certain events. The application uses the Flask-Mail extension to send
email. The application also uses Celery to send email asynchronously so
that there is no delayed response to the client. The dependencies are
simply Flask, Flask-Mail, and Celery 3.0.
I added a gist that outlines the basic contours of this scenario:
https://gist.github.com/3201722. I've used (very basic) documented patterns
for application factories<http://flask.pocoo.org/docs/patterns/appfactories/>,
blueprints <http://flask.pocoo.org/docs/blueprints/#blueprints>, and dedicated
If you download the gist, you can run Flask using "python run.py" and
Celery using "celery worker -A application". Ping the only available route
The primary issue is that Flask-Mail no longer works. (I have a similar
setup in production using Flask-Celery with Celery 2.5.5 and it does work.)
The exception traceback is:
Traceback (most recent call last):
line 212, in trace_task
R = retval = fun(*args, **kwargs)
File "/Users/jzempel/Sites/temp/application/tasks.py", line 8, in
message = Message("Test")
line 58, in __init__
app = _request_ctx_stack.top.app
AttributeError: 'NoneType' object has no attribute 'app'
So it appears that Celery 3.0 (unlike Flask-Celery & Celery 2.5.5) does not
allow task access to the application context. It seems like this would be a
common requirement, especially when an app wants to depend on Flask
extensions in an asynchronous process.
Here is a second gist, modified in the way that I wish a Flask-Celery 3.0
extension would work with Flask: https://gist.github.com/3201883. The main
differences are: 1) no more celery.py, 2) Celery is instantiated in
extensions.py, 3) Celery is initialized (implicitly depending on current
application config) in the "create_app" application factory. And (somehow?)
the ability to reference the application context in an async celery task is
Does this help clarify what I'm seeing and why I feel there is still an
important niche for Flask-Celery?
On Thu, Jul 26, 2012 at 8:12 AM, Ask Solem <email@example.com> wrote:
> Hey Jonathan,
> Sorry for the late reply but I just found your message
> in a Google search.
> >With my current config, Celery 3.0 is temporarily broken. I'm finding that
> >it's taking a spaghetti of dependencies to get it right. As such, it
> >certainly feels to me like there is a continued need for the extension. In
> >my mind a Flask-Celery extension would (at least) do the following:
> What sort of dependencies? The only dependency you should need
> is the 'celery' package itself.
> >1) Accept an instance of a Flask application in either an extension
> >constructor or init_app() method (i.e. understands how to operate with a
> >deferred configuration)
> What sort of things would you like Celery do do with the Flask application?
> Is there anything it needs beyond the configuration?
> You can tell Celery to use a Flask apps configuration easily:
> app = Flask()
> celery = Celery()
> add_defaults will be part of 3.0.4, it's different from conf.update in that
> it won't make a copy of your configuration, and it won't have to pickle it
> when starting new child processes. For version before you have to do:
> celery.conf.defaults.insert(0, app.config)
> celery.conf._order.insert(1, app.config)
> or just:
> celery.conf.update(app.config) # makes a copy
> In this case the config will be read directly, but you
> can pass a callable to have it loaded lazily:
> celery.add_defaults(lambda: flask_app.config)
> >2) Support a common Flask/Celery configuration (i.e. no celeryconfig.py)
> Celery 3.0 does not require a celeryconfig, as noted above you can
> use the Flask configuration, use a separate config module for Celery,
> or specify the celery configuration directly in the code.
> >3) Operate well when instantiated from an Application
> Could you elaborate on this?
> >4) Continue to provide working Flask-Script commands that can be used
> >during development
> But with the new 'celery' umbrella command why would you need it?
> All you need to do is specify the celery instance that should be used:
> celery -A myapp worker -l info
> (-A myapp is a shortcut to --app=myapp.celery, see
> You can also easily start the umbrella command programmatically:
> celery = Celery('myapp')
> if __name__ == '__main__':
> where 'myapp' is the name of the module, necessary to convert the __main__
> when generating task names.
> Flask-Celery already consisted of very few lines of code when used with
> Celery 2.5,
> where most of the code was there to convert optparse options into argparse
> in Celery 3.1 we hope to move to argparse, and there's an issue open for
> it too:
> I believe that everyone sharing the same API is a big step forward, with
> advantages like having the same documentation and issue tracker.
> Of course there will be some issues to solve at first, but I believe
> we should be able to fix them and improve the core API so that everyone
> is happy.
> And if not, and we find out that it was better after all,
> we can always go back to using extensions...
> Ask Solem
> twitter.com/asksol | +44 (0)7713357179