librelist archives

Instance Folders in Flask 0.8

From:

Armin Ronacher

Date:

2011-08-10 @ 12:03

Hi,

This has been on my mind for a long time and I finally made the decision
to introduce it into Flask.  I am not entirely happy with the name yet,
so maybe that should be changed.  Also it's not very prominent in the
docs so far.

Basically we all know the problem that config files differ in various
environments (locally and in production).  Additionally not only the
database is different, we also have to store some other files on the
filesystem that might change (uploads etc.)

Dropping them directly into Flask.root_path is not a clever idea since
that is often a package and under version control.  Flask 0.8 now has a
new path (Flask.instance_path) which defaults to `instance` next to your
package or module where you can drop this kind of deployment specific
stuff in: configs, uploaded files, cached content etc.

This also makes it easier for flask extensions that work with the file
system to select useful defaults (imagine file system sessions etc.).
Instead of having to ask for the path they could select a name in the
instance folder and drop it there (instance/flaskext.fs-session) etc.

Configs can also be set to work relative to the instance path rather
than the root path.  Here is an example of how you can configure the
application in Flask 0.8 to load the config defaults from
yourapplication.default_settings and then override the values there with
the contents of the appconfig.py file in the instance folder if it exists:

app = Flask(__name__, instance_relative_config=True)
app.config.from_object('yourapplication.default_settings')
app.config.from_pyfile('appconfig.py', silent=True)

What are your thoughts on this?


Regards,
Armin

Hi,

I just updated the detection algorithm for the instance path.  If the
application is installed into a virtualenv or the global Python it will
find the proper prefix and drop it into share/appname-instance.

For example you have an app myapp installed into /usr/local:

Package:
  /usr/local/lib/python2.6/site-packages/myapp
Instance path:
  /usr/local/share/myapp-instance

Same with any virtualenv

Package:
  /var/www/myhost.com/env/lib/python2.7/site-packages/myapp
Instance path:
  /var/www/myhost.com/env/share/myapp-instance

If it's not installed at all it's dropped as 'instance' next to the package:

Package:
  /home/username/dev/myapp/myapp
Instance path:
  /home/username/dev/myapp/instance


Regards,
Armin

On Aug 10, 2011, at 6:58 PM, Armin Ronacher wrote:

> I just updated the detection algorithm for the instance path.  If the
> application is installed into a virtualenv or the global Python it will
> find the proper prefix and drop it into share/appname-instance.

I would suggest "var" instead of "share", since it's storing runtime 
variable data, not static shared resources. As for the name, "instance" is
what Zope 2 called a folder with a similar purpose. Perhaps "runtime 
folder" would be clearer?

Other than that, great, I don't have to implement the instance folder 
myself if it's part of the framework :)

Cheers,
-- Alex

> If it's not installed at all it's dropped as 'instance' next to the package:
> 
> Package:
>  /home/username/dev/myapp/myapp
> Instance path:
>  /home/username/dev/myapp/instance

What's the definition of "instance path"? The docs are a little vague and 
self-referential in their description. Is it the synonym for the absolute 
path of the cwd where the application was started?

Outside of open_instance_resource(), are there other subtle uses for this 
(or future uses)? As an example, are there plans to integrate auto-loading
of config variables, such as load_instance_config()? Right now do I do 
this via a local import, but trying to figure out if there's going to be 
some machinery coming down the pipe.

tia. -sc



--
Sean Chittenden
sean@chittenden.org

Hi,

On 8/10/11 6:19 PM, Sean Chittenden wrote:
> What's the definition of "instance path"? The docs are a little vague and
> self-referential in their description. Is it the synonym for the
> absolute path of the cwd where the application was started?
There is no such thing as a working directory in a WSGI web app.  The
definition of the instance currently is a folder named instance next to
your package or module or a folder named appname-instance in the share
folder of your prefix if installed.

> Outside of open_instance_resource(), are there other subtle uses for
> this (or future uses)? As an example, are there plans to integrate
> auto-loading of config variables, such as load_instance_config()?
> Right now do I do this via a local import, but trying to figure out
> if there's going to be some machinery coming down the pipe.
The instance folder is especially useful for extensions that need a
place on the filesystem to dump stuff (sessions, OpenID nonce store,
uploaded files, etc.)


Regards,
Armin

>> What's the definition of "instance path"? The docs are a little vague and
>> self-referential in their description. Is it the synonym for the
>> absolute path of the cwd where the application was started?
> There is no such thing as a working directory in a WSGI web app.  The
> definition of the instance currently is a folder named instance next to
> your package or module or a folder named appname-instance in the share
> folder of your prefix if installed.

So Flask apps chdir(2) to / and then access all content not resident in 
memory by using absolute paths? Or is the pwd for a given PID somewhat 
arbitrary in that it's unchanged from the processes startup? For some 
reason I had it in my mind that it chdir(2)'ed to the base directory of a 
given instance (i.e. /home/user/src/my_app).

-sc

--
Sean Chittenden
sean@chittenden.org

Hi,

On 2011-08-10 9:19 PM, Sean Chittenden wrote:
> So Flask apps chdir(2) to / and then access all content not resident in
 > memory by using absolute paths? Or is the pwd for a given PID
 > somewhat arbitrary in that it's unchanged from the processes startup?
Flask internally never uses relative paths with the notable exception of 
support for the Python interactive interpreter.  It checks for the 
location of the module that creates the Flask instance (which is why 
__main__ is passed to the constructor usually).

Relative paths are a problem because at any point the cwd could change 
(from the outside, because a library does that, because something else 
in the same process does it (consider embedded python in combination 
with a PHP app etc.)).


Regards,
Armin

 I like the idea and the concept, but the name isn't very intuitive. Maybe
we can find a better name for it? 

Thanks for your efforts.
Cheers Peter

On Wednesday, August 10, 2011 at 7:07 PM, Armin Ronacher wrote:

> Hi,
> 
> On 8/10/11 6:19 PM, Sean Chittenden wrote:
> > What's the definition of "instance path"? The docs are a little vague and
> > self-referential in their description. Is it the synonym for the
> > absolute path of the cwd where the application was started?
> There is no such thing as a working directory in a WSGI web app. The
> definition of the instance currently is a folder named instance next to
> your package or module or a folder named appname-instance in the share
> folder of your prefix if installed.
> 
> > Outside of open_instance_resource(), are there other subtle uses for
> > this (or future uses)? As an example, are there plans to integrate
> > auto-loading of config variables, such as load_instance_config()?
> > Right now do I do this via a local import, but trying to figure out
> > if there's going to be some machinery coming down the pipe.
> The instance folder is especially useful for extensions that need a
> place on the filesystem to dump stuff (sessions, OpenID nonce store,
> uploaded files, etc.)
> 
> 
> Regards,
> Armin

On 10 August 2011 22:03, Armin Ronacher <armin.ronacher@active-4.com> wrote:

> Hi,
>
> This has been on my mind for a long time and I finally made the decision
> to introduce it into Flask.  I am not entirely happy with the name yet,
> so maybe that should be changed.  Also it's not very prominent in the
> docs so far.
>
> Basically we all know the problem that config files differ in various
> environments (locally and in production).  Additionally not only the
> database is different, we also have to store some other files on the
> filesystem that might change (uploads etc.)
>
> Dropping them directly into Flask.root_path is not a clever idea since
> that is often a package and under version control.  Flask 0.8 now has a
> new path (Flask.instance_path) which defaults to `instance` next to your
> package or module where you can drop this kind of deployment specific
> stuff in: configs, uploaded files, cached content etc.
>

Unless I’ve misunderstood this, instance_path would default to a path within
the package, so if you installed this into a virtualenv, this would point
somewhere like
"/path/to/virtualenv/lib/pythonX.Y/site-packages/myapp/instance", yes?
Why would that be a sensible default for the instance_path?
Would not the current working directory be a more useful default in
practice?
(I think it would be, but perhaps there’s some reason why it is a bad idea…)

This also makes it easier for flask extensions that work with the file
> system to select useful defaults (imagine file system sessions etc.).
> Instead of having to ask for the path they could select a name in the
> instance folder and drop it there (instance/flaskext.fs-session) etc.
>
> Configs can also be set to work relative to the instance path rather
> than the root path.  Here is an example of how you can configure the
> application in Flask 0.8 to load the config defaults from
> yourapplication.default_settings and then override the values there with
> the contents of the appconfig.py file in the instance folder if it exists:
>
> app = Flask(__name__, instance_relative_config=True)
> app.config.from_object('yourapplication.default_settings')
> app.config.from_pyfile('appconfig.py', silent=True)
>

Does it really make sense to enable this behaviour on the Flask object?
Surely it should just be an option when using one of the app.config.from_*()
methods?

app = Flask(__name__)
app.config.from_object('yourapplication.default_settings')
app.config.from_pyfile('appconfig.py', instance_relative=True, silent=True)

Regards,
  Peter


> What are your thoughts on this?
>
>
> Regards,
> Armin
>



-- 
Peter Ward
http://flowblok.id.au/
BIT III, Sydney University

Hi,

On 8/10/11 2:37 PM, Peter Ward wrote:
> Unless I’ve misunderstood this, instance_path would default to a path
> within the package, so if you installed this into a virtualenv, this
> would point somewhere like
> "/path/to/virtualenv/lib/pythonX.Y/site-packages/myapp/instance", yes?
Currently yes.  I will see if I can find a better default for installed
packages.

> Why would that be a sensible default for the instance_path?
> Would not the current working directory be a more useful default in
> practice?
Well.  The working directory cannot be used portable in web applications
so that is something to step away from.

> Does it really make sense to enable this behaviour on the Flask object?
> Surely it should just be an option when using one of the
> app.config.from_*() methods?
This is independent of the config.


Regards,
Armin

 +100 for relative paths for configurations. I'd *love* to be able to keep
that wsgi file, for example, under version control w/ a simple convention.


--  
Kenneth Reitz


On Wednesday, August 10, 2011 at 10:01 AM, Armin Ronacher wrote:

> Hi,
>  
> On 8/10/11 2:37 PM, Peter Ward wrote:
> > Unless I’ve misunderstood this, instance_path would default to a path
> > within the package, so if you installed this into a virtualenv, this
> > would point somewhere like
> > "/path/to/virtualenv/lib/pythonX.Y/site-packages/myapp/instance", yes?
> Currently yes. I will see if I can find a better default for installed
> packages.
>  
> > Why would that be a sensible default for the instance_path?
> > Would not the current working directory be a more useful default in
> > practice?
> Well. The working directory cannot be used portable in web applications
> so that is something to step away from.
>  
> > Does it really make sense to enable this behaviour on the Flask object?
> > Surely it should just be an option when using one of the
> > app.config.from_*() methods?
> This is independent of the config.
>  
>  
> Regards,
> Armin