librelist archives

« back to archive

Database bootstrapping

Database bootstrapping

From:
Adam Davis
Date:
2012-07-14 @ 17:54
I am writing my first flask application, using SQL Alchemy. There is some 
data required for the application to function that I want to have 
automatically inserted whenever a new database is created (eg when running
tests, and when updating the schema and recreating the databse of my 
development instance). I have searched for patterns about how to do this 
sort of thing, but haven't found much that's relevant yet.

I have tried including this bootstrapping functionality in:

- my database module (where init_db is defined and the engine and database
session are configured.) When I try defining and calling the bootstrapping
code here, there is a problem with importing the class for the model of 
the data because declarative_base (defined in my models module) is then 
already in the namespace of my 'database' module.

- the script which starts the flask application (similar to the pattern 
described here):

http://flask.pocoo.org/docs/patterns/packages/#simple-packages

Here, I am getting an error that my columns aren't unique (as I have 
defined a unique constraint on one of them). The same bootstrapping code 
works fine when I am testing the application, so I'm perplexed as to why 
it would be producing this error here.

Can anyone point me to patterns for doing bootstrapping of this sort?

Many thanks,

Adam

Re: [flask] Database bootstrapping

From:
Simon Sapin
Date:
2012-07-16 @ 07:56
Le 14/07/2012 19:54, Adam Davis a écrit :
> I am writing my first flask application, using SQL Alchemy. There is
> some data required for the application to function that I want to
> have automatically inserted whenever a new database is created (eg
> when running tests, and when updating the schema and recreating the
> databse of my development instance). I have searched for patterns
> about how to do this sort of thing, but haven't found much that's
> relevant yet.
>
> I have tried including this bootstrapping functionality in:
>
> - my database module (where init_db is defined and the engine and
> database session are configured.) When I try defining and calling the
> bootstrapping code here, there is a problem with importing the class
> for the model of the data because declarative_base (defined in my
> models module) is then already in the namespace of my 'database'
> module.
>
> - the script which starts the flask application (similar to the
> pattern described here):
>
> http://flask.pocoo.org/docs/patterns/packages/#simple-packages
>
> Here, I am getting an error that my columns aren't unique (as I have
> defined a unique constraint on one of them). The same bootstrapping
> code works fine when I am testing the application, so I'm perplexed
> as to why it would be producing this error here.
>
> Can anyone point me to patterns for doing bootstrapping of this
> sort?

Hi,

In general, this stuff should go together with the script that creates 
the tables and other schema stuff. This script will be run for each test 
(on a separate testing database) or when you deploy the app, but maybe 
not every time the server is started.

Also, in debug mode the Werkzeug reloader creates new worker processes. 
This means that any import-time code will be run twice initially (in the 
parent and then first worker process) and then again on each reload.

Having stuff run twice might be why you get the "not unique" error.

The work-around is usually this:
http://flask.pocoo.org/docs/api/#flask.Flask.before_first_request

... but I think not in the case of database setup, which I would run 
only when deploying to a new machine.

Regards,
-- 
Simon Sapin

Re: [flask] Database bootstrapping

From:
Redouane Zait
Date:
2012-07-14 @ 18:14
i'd suggest you take a look at Flask-Script, here's an example i had laying
around :

from flask.ext.script import Manager, prompt_bool
from myapp import create_app, db

manager = Manager(create_app())

@manager.command
def createdb():
    """ creates all the database tables """
    db.create_all()

@manager.command
def dropdb():
    """ drops all the database tables """
    db.drop_all()

@manager.command
def filldb():
    """ fills the database with initial data"""
    with open('utils/data.sql', 'r') as f:
        sql = f.read()
        db.engine.execute(sql)

@manager.command
def freshdb():
    """ creates a fresh new database with initial data filled in
    """
    if prompt_bool('Are you sure you want a fresh db?'):
        dropdb()
        createdb()
        filldb()


you can adjust that to your needs, and use the script via command line
$ python manage.py dropdb


On Sat, Jul 14, 2012 at 6:54 PM, Adam Davis
<addavis@fosterandpartners.com>wrote:

> I am writing my first flask application, using SQL Alchemy. There is some
> data required for the application to function that I want to have
> automatically inserted whenever a new database is created (eg when running
> tests, and when updating the schema and recreating the databse of my
> development instance). I have searched for patterns about how to do this
> sort of thing, but haven't found much that's relevant yet.
>
> I have tried including this bootstrapping functionality in:
>
> - my database module (where init_db is defined and the engine and database
> session are configured.) When I try defining and calling the bootstrapping
> code here, there is a problem with importing the class for the model of the
> data because declarative_base (defined in my models module) is then already
> in the namespace of my 'database' module.
>
> - the script which starts the flask application (similar to the pattern
> described here):
>
> http://flask.pocoo.org/docs/patterns/packages/#simple-packages
>
> Here, I am getting an error that my columns aren't unique (as I have
> defined a unique constraint on one of them). The same bootstrapping code
> works fine when I am testing the application, so I'm perplexed as to why it
> would be producing this error here.
>
> Can anyone point me to patterns for doing bootstrapping of this sort?
>
> Many thanks,
>
> Adam
>
>
>

Re: [flask] Database bootstrapping

From:
Adam Davis
Date:
2012-07-14 @ 18:40
Thank you for the reply. From a quick look at the docs for flask-script, 
I'm not sure it's what I'm looking for (or at least not what I thought I 
was looking for): as the docs say, the extension is for "tasks that belong
outside the web application itself." I am looking to integrate this 
functionality into the web app (if critical data isn't present when I 
execute the app runner, it should be inserted) and the test suite for that
app. I would ideally like to have this functionality modularized so it can
be run from both development/production instances and tests, automatically
on setup.

Flask-script seems at first glance more intended for manual 
maintenance/admin tasks (executed from the command line) than repeated 
actions invoked in the course of running and testing the application -- or
am I missing the point? Are there examples of using flask-script in/for 
tests? I'm using nose, FWIW, and it's not clear to me how I would 
incorporate a flask-script based solution (geared to command line usage) 
with the nosetests executable, also run from the command line.

-Adam


-----Original Message-----
From: flask@librelist.com on behalf of Redouane Zait
Sent: Sat 14/07/2012 7:14 PM
To: flask@librelist.com
Subject: Re: [flask] Database bootstrapping
 
i'd suggest you take a look at Flask-Script, here's an example i had 
laying around :

from flask.ext.script import Manager, prompt_bool
from myapp import create_app, db

manager = Manager(create_app())

@manager.command
def createdb():
    """ creates all the database tables """
    db.create_all()

@manager.command
def dropdb():
    """ drops all the database tables """
    db.drop_all()

@manager.command
def filldb():
    """ fills the database with initial data"""
    with open('utils/data.sql', 'r') as f:
        sql = f.read()
        db.engine.execute(sql)

@manager.command
def freshdb():
    """ creates a fresh new database with initial data filled in
    """
    if prompt_bool('Are you sure you want a fresh db?'):
        dropdb()
        createdb()
        filldb()


you can adjust that to your needs, and use the script via command line
$ python manage.py dropdb


On Sat, Jul 14, 2012 at 6:54 PM, Adam Davis <addavis@fosterandpartners.com> wrote:


	I am writing my first flask application, using SQL Alchemy. There is some
data required for the application to function that I want to have 
automatically inserted whenever a new database is created (eg when running
tests, and when updating the schema and recreating the databse of my 
development instance). I have searched for patterns about how to do this 
sort of thing, but haven't found much that's relevant yet.
	
	I have tried including this bootstrapping functionality in:
	
	- my database module (where init_db is defined and the engine and 
database session are configured.) When I try defining and calling the 
bootstrapping code here, there is a problem with importing the class for 
the model of the data because declarative_base (defined in my models 
module) is then already in the namespace of my 'database' module.
	
	- the script which starts the flask application (similar to the pattern 
described here):
	
	http://flask.pocoo.org/docs/patterns/packages/#simple-packages
	
	Here, I am getting an error that my columns aren't unique (as I have 
defined a unique constraint on one of them). The same bootstrapping code 
works fine when I am testing the application, so I'm perplexed as to why 
it would be producing this error here.
	
	Can anyone point me to patterns for doing bootstrapping of this sort?
	
	Many thanks,
	
	Adam