This may be too specific to be an actual extension but I was unhappy with SQLAlchemy-Migrate and just wanted something simple and predictable for dealing with schema migration with Flask/SQLAlchemy setups. Much of the code is from the SimpleMigrations project for Django. I've only tested it on Postgres but the current stuff is so simple it should work on MySQL as well. I have plans to add more functionality, break up into multiple files, etc but want to keep it pretty simple. It needs better docs, but I just wanted to get it out to see what people thought. It's pretty simple so if you are interested just check the code. Patches welcome. https://github.com/adamrt/flask-migrate Sorry for the short/bad email, it's pretty late over here. :) Adam Patterson
Hi, +1 for effort at the very least. I will have a look at the implementation later today, might need some docs :) Why not base it on the SQLAlchemy migration package? Also I think some basic DDL support landed in SQLAlchemy or is supposed to land, so maybe have a look at that too. Especially sqlite is really hard to migrate. The only native alter table operation it supports is adding columns so for most operations you need temporary tables. Regards, Armin
On Mon, May 30, 2011 at 9:43 AM, Armin Ronacher <armin.ronacher@active-4.com> wrote: > Hi, > > Why not base it on the SQLAlchemy migration package? I was trying to deal with SQLAlchemy-Migrate for a while now and for my needs its just too much. Nearly 50 python files. It obviously does much more but my requirements are small. I just became extremely frustrated with it. >Also I think some basic DDL support landed in SQLAlchemy > or is supposed to land, so maybe have a look at that too. Thanks, I'll definitely look into this. > Especially sqlite is really hard to migrate. The only native alter > table operation it supports is adding columns so for most operations you > need temporary tables. I'm only using Postgres right now, but I'm happy to add support for others as well. I need to checkout out South/SQLAlchemy-Migrate to see how they are dealing with SQLite
On May 30, 2011, at 11:39 AM, Adam Patterson wrote: > >> Especially sqlite is really hard to migrate. The only native alter >> table operation it supports is adding columns so for most operations you >> need temporary tables. > I'm only using Postgres right now, but I'm happy to add support for > others as well. I need to checkout out South/SQLAlchemy-Migrate to see > how they are dealing with SQLite My advice for SQLite would be to *don't* deal with it. SQLite has almost no support for ALTER. Applications that choose to use SQLite should be prepared to not support live migrations, and should instead be able to rebuild an entire database. If you're looking for more ideas re: migration libs for SQLAlchemy, I have my own project called Alembic https://bitbucket.org/zzzeek/alembic . Its extremely early but it does work and I've started integrating it with my current work gig as a test. I haven't had time to document it yet which is a key reason I can't really put it out, but it improves upon various issues I've had with SQLAlchemy-migrate. It allows short commands like alter_column(tablename, colname, type=new_type) which create minimal Table/Column metadata behind the scenes for you. It also allows the developer full control over the establishment and usage of the Engine, thus allowing transactional DDL, multiple engine support etc. If someone wants to help document it and maybe help me with development, I'd put it up on pypi.
Hey Michael, I read your mentions of Alembic around the internet when searching for an alternative to SQLAlchemy-Migrate. I'll definitely check it out today. Also, I renamed my tiny thing to flask-evolution for now. its location is now at https://github.com/adamrt/flask-evolution On Mon, May 30, 2011 at 1:38 PM, Michael Bayer <mike_mp@zzzcomputing.com> wrote: > > On May 30, 2011, at 11:39 AM, Adam Patterson wrote: > > Especially sqlite is really hard to migrate. The only native alter > > table operation it supports is adding columns so for most operations you > > need temporary tables. > > I'm only using Postgres right now, but I'm happy to add support for > others as well. I need to checkout out South/SQLAlchemy-Migrate to see > how they are dealing with SQLite > > My advice for SQLite would be to *don't* deal with it. SQLite has almost no > support for ALTER. Applications that choose to use SQLite should be > prepared to not support live migrations, and shoul d instead be able to > rebuild an entire database. > If you're looking for more ideas re: migration libs for SQLAlchemy, I have > my own project called Alembic https://bitbucket.org/zzzeek/alembic . Its > extremely early but it does work and I've started integrating it with my > current work gig as a test. I haven't had time to document it yet which is > a key reason I can't really put it out, but it improves upon various issues > I've had with SQLAlchemy-migrate. It allows short commands like > alter_column(tablename, colname, type=new_type) which create minimal > Table/Column metadata behind the scenes for you. It also allows the > developer full control over the establishment and usage of the Engine, thus > allowing transactional DDL, multiple engine support etc. > If someone wants to help document it and maybe help me with development, I'd > put it up on pypi. > > >
Hi, On 5/30/11 5:39 PM, Adam Patterson wrote: > I was trying to deal with SQLAlchemy-Migrate for a while now and for > my needs its just too much. Nearly 50 python files. It obviously does > much more but my requirements are small. I just became extremely > frustrated with it. In the case where a Flask extension does not simply wrap a framework independent library like SQLAlchemy-Migrate I would prefer if that extension would not name itself overly generic like "migrate". Maybe name it Flask-SimpleMigrate or something. Regards, Armin
On Mon, May 30, 2011 at 11:01 AM, Armin Ronacher <armin.ronacher@active-4.com> wrote: > Hi, > In the case where a Flask extension does not simply wrap a framework > independent library like SQLAlchemy-Migrate I would prefer if that > extension would not name itself overly generic like "migrate". Maybe > name it Flask-SimpleMigrate or something. > Hah, I had a sentence in my previous email mentioning that its probably not appropriate to use flask-migrate as a name but just tossed it at the last second. I complete agree and I'll change it.
On Mon, May 30, 2011 at 11:39 AM, Adam Patterson <fakeempire@gmail.com> wrote: > On Mon, May 30, 2011 at 9:43 AM, Armin Ronacher > <armin.ronacher@active-4.com> wrote: >> Why not base it on the SQLAlchemy migration package? > I was trying to deal with SQLAlchemy-Migrate for a while now and for > my needs its just too much. Nearly 50 python files. It obviously does > much more but my requirements are small. I just became extremely > frustrated with it. I definitely see a use in a simple migration framework for a Flask extension. I had a similar experience on my first foray in SQLAlchemy-Migrate. But, you could base an extension on SQLAlchemy-Migrate and provide a simpler interface for Flask users. Because... >>Also I think some basic DDL support landed in SQLAlchemy >> or is supposed to land, so maybe have a look at that too. > Thanks, I'll definitely look into this. > >> Especially sqlite is really hard to migrate. The only native alter >> table operation it supports is adding columns so for most operations you >> need temporary tables. > I'm only using Postgres right now, but I'm happy to add support for > others as well. I need to checkout out South/SQLAlchemy-Migrate to see > how they are dealing with SQLite This is a non-trivial problem. :-) I for one am glad to see you working on it. I have some very specific migration needs on one of my projects, and have been experimenting a bit (it's not top priority yet on my end, sorry!). -Ron
On Mon, May 30, 2011 at 4:52 AM, Adam Patterson <fakeempire@gmail.com> wrote: > This may be too specific to be an actual extension I think it fits well as an extension. > Much of the code is from the SimpleMigrations project for > Django. I've only tested it on Postgres but the current stuff is so > simple it should work on MySQL as well. I have plans to add more > functionality, break up into multiple files, etc but want to keep it > pretty simple. Looks very neat! I'm a little paranoid about database migrations, but I'll note to try test out on one of my databases. The SQL in your extension looks vanilla, but I'm curious to know if SQLAlchemy itself has apis to deal with dialects for migrations. -Ron
> Looks very neat! I'm a little paranoid about database migrations, but > I'll note to try test out on one of my databases. Me too. Thats why I wanted something thats simple and predictable. Where no magic would happen. I was really looking for a nice way to keep my dev/staging/production db servers in sync. >The SQL in your extension looks vanilla, > but I'm curious to know if SQLAlchemy itself > has apis to deal with dialects for migrations. I was looking for that myself to get the portability of SQLAlchemy for free but I was unable to find anything in my (light) search. I'll continue to check and feel free to let me know if you find anything. Thanks!