librelist archives

Registering extensions with the app object

From:

Michael Elsdörfer

Date:

2010-08-23 @ 18:29

I'm working on a set of related, Flask-based applications. I'm putting 
shared code, like the authentication code, into what are essentially 
private Flask extensions.

As the number of dependencies between the different extensions increase, 
passing around the various extension objects gets quite tedious.

I find that it's convenient to store the extensions with the app object, 
e.g. Flask-SQLAlchemy is app.db, Flask-Assets is app.assets, my auth 
"extension" is app.auth (and I can access the user model as app.auth.User)

This is all very straightforward, except for two concerns:

* I worry I might accidentally override an actual attribute of the Flask 
app object.

* I'm not quite fond of the fact that my applications are responsible of 
setting up these attributes, and my extensions just silently expect them 
to exist. My auth extension expects the access the database via 
"app.db", but I have to remember to assign theses connections correctly 
in my app.

I wonder if some sort of protocol could be designed to put the burden on 
the extension to connect itself to the app object for reverse access, 
and to possibly provide a space there for it to do so.

E.g. Flask-SQLAlchemy would set maybe set "app.ext.sqlalchemy".

I realize that for some, if not most extensions it can make perfect 
sense to create multiple instances of the extension object. I'm not 
really sure how to deal with that problem, but still, but maybe somebody 
else has an idea. I just wanted to put some thoughts on this use-case 
out there.

Michael

On 23 August 2010 19:29, Michael Elsdörfer <michael@elsdoerfer.com> wrote:
> I'm working on a set of related, Flask-based applications. I'm putting
> shared code, like the authentication code, into what are essentially
> private Flask extensions.
>

Why extensions ? Extensions are really just packages extending basic
Flask functionality, with some common standards, e.g. using the
flaskext namespace.

For your own code I'd suggest plain Python packages and modules, there
is no need for the added overhead.

> As the number of dependencies between the different extensions increase,
> passing around the various extension objects gets quite tedious.
>
> I find that it's convenient to store the extensions with the app object,
> e.g. Flask-SQLAlchemy is app.db, Flask-Assets is app.assets, my auth
> "extension" is app.auth (and I can access the user model as app.auth.User)
>
> This is all very straightforward, except for two concerns:
>
> * I worry I might accidentally override an actual attribute of the Flask
> app object.
>
> * I'm not quite fond of the fact that my applications are responsible of
> setting up these attributes, and my extensions just silently expect them
> to exist. My auth extension expects the access the database via
> "app.db", but I have to remember to assign theses connections correctly
> in my app.
>
> I wonder if some sort of protocol could be designed to put the burden on
> the extension to connect itself to the app object for reverse access,
> and to possibly provide a space there for it to do so.
>
> E.g. Flask-SQLAlchemy would set maybe set "app.ext.sqlalchemy".
>
> I realize that for some, if not most extensions it can make perfect
> sense to create multiple instances of the extension object. I'm not
> really sure how to deal with that problem, but still, but maybe somebody
> else has an idea. I just wanted to put some thoughts on this use-case
> out there.
>

For larger apps I have an extensions module which has for example:

db = SQLAlchemy()
mail = Mail()

and so on, and import and initialize them in my app factory function:

def create_app(config):
    ... set up
    db.init_app(app)
    return app

> Why extensions ? Extensions are really just packages extending basic
> Flask functionality, with some common standards, e.g. using the
> flaskext namespace.

Well, that's sort of what I need to do - extend Flask functionality. For
example, my auth code needs to install a before_request middleware to
resolve the current user. It also needs access to the Flask-SQLAlchemy
object in order to install the User model. So I need to write the code
as a callable which I can pass the app and db objects.

I'm not using the flaskext namespace, but other than that, it looks like
one.

> For larger apps I have an extensions module which has for example:

The "auth" code is shared between multiple, separate apps, so it doesn't
know from where to import the objects. I.e. it might be
"timetracker.extensions" or "mail.extensions" etc.

Michael

On 24 August 2010 12:03, Michael Elsdörfer <michael@elsdoerfer.com> wrote:
>> Why extensions ? Extensions are really just packages extending basic
>> Flask functionality, with some common standards, e.g. using the
>> flaskext namespace.
>
> Well, that's sort of what I need to do - extend Flask functionality. For
> example, my auth code needs to install a before_request middleware to
> resolve the current user. It also needs access to the Flask-SQLAlchemy
> object in order to install the User model. So I need to write the code
> as a callable which I can pass the app and db objects.
>

Create a "common" package shared between your apps, which includes the
User model and db instance.

So you can then do:

from common import db, auth

app = Flask(__name__)
db.init_app(app)
auth.init_app(app)

The db is just an instance of SQLAlchemy.

In the model code you can re-use the same db instance:

common/auth/models.py:

from common import db

class User(db.Model):
   ....



> I'm not using the flaskext namespace, but other than that, it looks like
> one.
>
>> For larger apps I have an extensions module which has for example:
>
> The "auth" code is shared between multiple, separate apps, so it doesn't
> know from where to import the objects. I.e. it might be
> "timetracker.extensions" or "mail.extensions" etc.
>
> Michael
>

> Create a "common" package shared between your apps, which includes the
> User model and db instance.

Hm. I'm not sure how I feel about the db-object being in the common
package, to be honest. Other than that, it's not so different from what
I'm doing:

from common import Auth
app.auth = Auth(app, db)

I still think it would be neat if the auth-setup method could defer the
db object through app, so it doesn't need to be passed separately.

BTW, another example is that my common auth code also provides a
create_user command for Flask-Script. Flask-Script makes the app object
available to commands, but for create_user I also need db. This means I
either need to go the way I am now (assigning to app.db), or put the
whole command implementation inside the scope of Auth().

Though I suppose your suggestion of moving the db object would work too,
of course.

Michael

You can create a db instance just for your auth app, and just initialize it:

from auth import db

class Auth(object):

   def init_app(self, app):
       db.init_app(app)

This *should* play nicely with other db instances, although I'd
confirm that with someone more knowledgeable
on the SQLAlchemy side of things.

For commands, I'd recommend creating a CreateUser command class that you
can then plug into your inividual app manager instance:

from auth.commands import CreateUser

manager = Manager(my_app)
manager.add_command(CreateUser())



On 24 August 2010 14:56, Michael Elsdörfer <michael@elsdoerfer.com> wrote:
>> Create a "common" package shared between your apps, which includes the
>> User model and db instance.
>
> Hm. I'm not sure how I feel about the db-object being in the common
> package, to be honest. Other than that, it's not so different from what
> I'm doing:
>
> from common import Auth
> app.auth = Auth(app, db)
>
> I still think it would be neat if the auth-setup method could defer the
> db object through app, so it doesn't need to be passed separately.
>
> BTW, another example is that my common auth code also provides a
> create_user command for Flask-Script. Flask-Script makes the app object
> available to commands, but for create_user I also need db. This means I
> either need to go the way I am now (assigning to app.db), or put the
> whole command implementation inside the scope of Auth().
>
> Though I suppose your suggestion of moving the db object would work too,
> of course.
>
> Michael
>

> I'm working on a set of related, Flask-based applications. I'm putting 
> shared code, like the authentication code, into what are essentially 
> private Flask extensions.
> 
> As the number of dependencies between the different extensions increase, 
> passing around the various extension objects gets quite tedious.
> 
> I find that it's convenient to store the extensions with the app object, 
> e.g. Flask-SQLAlchemy is app.db, Flask-Assets is app.assets, my auth 
> "extension" is app.auth (and I can access the user model as app.auth.User)
> 
> This is all very straightforward, except for two concerns:
> 
> * I worry I might accidentally override an actual attribute of the Flask 
> app object.
> 
> * I'm not quite fond of the fact that my applications are responsible of 
> setting up these attributes, and my extensions just silently expect them 
> to exist. My auth extension expects the access the database via 
> "app.db", but I have to remember to assign theses connections correctly 
> in my app.
> 
> I wonder if some sort of protocol could be designed to put the burden on 
> the extension to connect itself to the app object for reverse access, 
> and to possibly provide a space there for it to do so.
> 
> E.g. Flask-SQLAlchemy would set maybe set "app.ext.sqlalchemy".
> 
> I realize that for some, if not most extensions it can make perfect 
> sense to create multiple instances of the extension object. I'm not 
> really sure how to deal with that problem, but still, but maybe somebody 
> else has an idea. I just wanted to put some thoughts on this use-case 
> out there.
> 
> Michael
> 

I have suggested before an app.extensions dictionary (for simplicity,
but could be an object with attributes too) where extensions add
themselves on the slot based on their module or package name in the
flaskext package. Consistent, convenient and avoids name clashes.

I also suggested a flaskext function such that flaskext('assets') ==
current_app.extensions['assets'].

Some extensions such as my own Flask-Genshi already do attach themselves
to the app, but I feel bad doing this (somewhat monkey patchy) and it's
difficult to think of a good attribute name (I use "genshi_instance"
which I think is quite ugly).

Someone else suggested a store where extensions can store anything. I
think that proposal does not solve potential issues with name clashes,
there's no consistency in how to get an extension instance and you
likely still end up with bad names like "genshi_instance". If you want
to store many things associated with an app I suggest you store them on
your own instance somehow, ie.
flaskext('assets').extension_local_attribute.

One thing I think I want to avoid however is some kind of "extension
API"; better to just have a general and flexible enough app API that
extensions can use just as apps do.