librelist archives

« back to archive

Flask on GAE in 2012/13

Flask on GAE in 2012/13

From:
Paul Walsh
Date:
2013-01-25 @ 05:36
Does anyone have recent experience with Flask on GAE? I am about to start a
project that must sit on GAE.

We are undecided on using the non-relational data store, or using the new
CloudSQL offering.

Any advice, experience, tips based on recent deployments of Flask to GAE
are appreciated, thanks.


-- 
*Paul Walsh*
0543551144

Re: [flask] Flask on GAE in 2012/13

From:
Sean Lynch
Date:
2013-01-25 @ 06:30
I've deployed a few Flask-based apps on GAE, and finishing up my first
CloudSQL backed one (I have 3 using the Datastore and 1 on Cloud SQL).

If you plan to use Google Cloud SQL, I figure you'll use SQLAlchemy.  If
so, you'll want to look at a few of my StackOverflow questions and answers:

http://stackoverflow.com/questions/10763171/can-sqlalchemy-be-used-with-google-cloud-sql

http://stackoverflow.com/questions/14224679/unable-to-connect-to-google-cloud-sql-from-development-server-using-sqlalchemy

In short, SQLAlchemy supports Cloud SQL as of 0.7.8, but you'll want to use
my custom dialect (and 0.8) until the changes are merged into a SQLAlchemy
release. You can use the current (0.7.8 / 0.7.9) mysql+gaedbms:// dialect
to access Cloud SQL from a production App Engine instance, but if you want
to deploy your schema or have any kind of access to your Cloud SQL instance
from your development workstation you'll need the custom dialect.


If you use any extensions, you might run into this issue:
https://github.com/mitsuhiko/flask/issues/662

I'm using a lot of extensions (Flask-SQLAlchemy, Flask-Admin, Flask-Script,
Flask-Security, Flask-WTF, Flask-Debugtoolbar, and more :) ), and have
worked around any issues that have come up.  Most of the extensions can be
fixed by changing how imports are done (as Flask-Cache did -
https://github.com/thadeusb/flask-cache/issues/38), although most of the
extensions just work (once you put them in your sys.path and in the root of
your project, or as I have, in a packages folder with the following script:
https://gist.github.com/4632192).

Hopefully the Flask issue 662 is resolved (it seems to be an easy fix, but
I haven't had time to submit a pull request and write some tests to verify).

With Cloud SQL, I do most of my dev using a local sqlite and later MySQL
instance, and used the Flask/Werkzeug development server (and not the GAE
dev server as I tried to not have any dependences on google APIs (mail,
etc)).

Let me know if I can help you out with any issues.  I've been planning to
create an extension to help with some GAE-specific items (flask-script
command to copy virtualenv packages to app's packages/ directory,
flask-script modified shell using the remote_api API to access the
Datastore remotely, etc).

On Fri, Jan 25, 2013 at 12:36 AM, Paul Walsh <paulywalsh@gmail.com> wrote:

> Does anyone have recent experience with Flask on GAE? I am about to start
> a project that must sit on GAE.
>
> We are undecided on using the non-relational data store, or using the new
> CloudSQL offering.
>
> Any advice, experience, tips based on recent deployments of Flask to GAE
> are appreciated, thanks.
>
>
> --
> *Paul Walsh*
> 0543551144
>
>

Re: Flask on GAE in 2012/13

From:
Paul Walsh
Date:
2013-01-25 @ 07:12
Thanks Sean.

You've provided a wealth of info right here that puts me on good footing.

Is there any significant reason you choose CloudSQL for your latest
project, and not the Datastore?


On Friday, January 25, 2013, Sean Lynch wrote:

> I've deployed a few Flask-based apps on GAE, and finishing up my first
> CloudSQL backed one (I have 3 using the Datastore and 1 on Cloud SQL).
>
> If you plan to use Google Cloud SQL, I figure you'll use SQLAlchemy.  If
> so, you'll want to look at a few of my StackOverflow questions and answers:
>
> 
http://stackoverflow.com/questions/10763171/can-sqlalchemy-be-used-with-google-cloud-sql
>
> 
http://stackoverflow.com/questions/14224679/unable-to-connect-to-google-cloud-sql-from-development-server-using-sqlalchemy
>
> In short, SQLAlchemy supports Cloud SQL as of 0.7.8, but you'll want to
> use my custom dialect (and 0.8) until the changes are merged into a
> SQLAlchemy release. You can use the current (0.7.8 / 0.7.9)
> mysql+gaedbms:// dialect to access Cloud SQL from a production App Engine
> instance, but if you want to deploy your schema or have any kind of access
> to your Cloud SQL instance from your development workstation you'll need
> the custom dialect.
>
>
> If you use any extensions, you might run into this issue:
> https://github.com/mitsuhiko/flask/issues/662
>
> I'm using a lot of extensions (Flask-SQLAlchemy, Flask-Admin,
> Flask-Script, Flask-Security, Flask-WTF, Flask-Debugtoolbar, and more :) ),
> and have worked around any issues that have come up.  Most of the
> extensions can be fixed by changing how imports are done (as Flask-Cache
> did - https://github.com/thadeusb/flask-cache/issues/38), although most
> of the extensions just work (once you put them in your sys.path and in the
> root of your project, or as I have, in a packages folder with the following
> script: https://gist.github.com/4632192).
>
> Hopefully the Flask issue 662 is resolved (it seems to be an easy fix, but
> I haven't had time to submit a pull request and write some tests to verify).
>
> With Cloud SQL, I do most of my dev using a local sqlite and later MySQL
> instance, and used the Flask/Werkzeug development server (and not the GAE
> dev server as I tried to not have any dependences on google APIs (mail,
> etc)).
>
> Let me know if I can help you out with any issues.  I've been planning to
> create an extension to help with some GAE-specific items (flask-script
> command to copy virtualenv packages to app's packages/ directory,
> flask-script modified shell using the remote_api API to access the
> Datastore remotely, etc).
>
> On Fri, Jan 25, 2013 at 12:36 AM, Paul Walsh 
<paulywalsh@gmail.com<javascript:_e({}, 'cvml', 'paulywalsh@gmail.com');>
> > wrote:
>
>> Does anyone have recent experience with Flask on GAE? I am about to start
>> a project that must sit on GAE.
>>
>> We are undecided on using the non-relational data store, or using the
>> new CloudSQL offering.
>>
>> Any advice, experience, tips based on recent deployments of Flask to GAE
>> are appreciated, thanks.
>>
>>
>> --
>> *Paul Walsh*
>> 0543551144
>>
>>
>

-- 
*Paul Walsh*
0543551144

Re: [flask] Re: Flask on GAE in 2012/13

From:
Sean Lynch
Date:
2013-01-25 @ 13:51
We chose Cloud SQL for our latest project for a few reasons:

  - This project had a large reporting requirement, which would have been
difficult without denormalizing our data a lot, or running a lot of map
reduce jobs.  I considered using a hybrid approach of powering the app
using the Datastore for speed / efficiency, but also writing to Cloud SQL
for reporting (or to BigQuery, which would have been cheaper), but in the
end, we decided it wasn't worth adding extra complexity and just use Cloud
SQL completely.

  - Along with reports, the app had a role-centric view that all data (and
related data) visibility was based on what your role was (and also what
associations you had). This greatly complicated how we would be able to
store the data, especially with the restrictions on querying (

https://developers.google.com/appengine/docs/python/datastore/queries#Restrictions_on_Queries
)

  - My team for this project didn't have any experience with the Datastore
or map reduce (and I'm still working on getting a good grasp of the
algorithms/techniques/best practices), so to keep me from having to do the
lion share of the data acces / report work, I chose Cloud SQL.

It's nice you get 1GB of free storage on Datastore, but as with my other
projects, it's the number of operations you're typically paying the most
for.  You have to spend a lot of time optimizing your indexes and
extensively using memcache (which the newer `ndb` package helps a lot).

Cloud SQL currently offers 1 free D0 instance till June, so it should give
you enough time to try it out.  I wrote this app making sure I could host
it elsewhere (AWS / Heroku) in case I ran into any issues, but so far I've
been able to work through the (the SQLAlchemy dialect almost made me lose
some sleep until I figured out what was going on).  Once you do setup a
paid instance, it's nice you can setup a "dynamic" Per-Use instance like
how App Engine instances work, where the instance shuts off and starts up
automatically (after 15 minutes of usage).  You can also get a package
plan, where you pay a fixed price per day, and the instances will stay up
for 12 hours of inactivity.


On Fri, Jan 25, 2013 at 2:12 AM, Paul Walsh <paulywalsh@gmail.com> wrote:

> Thanks Sean.
>
> You've provided a wealth of info right here that puts me on good footing.
>
> Is there any significant reason you choose CloudSQL for your latest
> project, and not the Datastore?
>
>
> On Friday, January 25, 2013, Sean Lynch wrote:
>
>> I've deployed a few Flask-based apps on GAE, and finishing up my first
>> CloudSQL backed one (I have 3 using the Datastore and 1 on Cloud SQL).
>>
>> If you plan to use Google Cloud SQL, I figure you'll use SQLAlchemy.  If
>> so, you'll want to look at a few of my StackOverflow questions and answers:
>>
>> 
http://stackoverflow.com/questions/10763171/can-sqlalchemy-be-used-with-google-cloud-sql
>>
>> 
http://stackoverflow.com/questions/14224679/unable-to-connect-to-google-cloud-sql-from-development-server-using-sqlalchemy
>>
>> In short, SQLAlchemy supports Cloud SQL as of 0.7.8, but you'll want to
>> use my custom dialect (and 0.8) until the changes are merged into a
>> SQLAlchemy release. You can use the current (0.7.8 / 0.7.9)
>> mysql+gaedbms:// dialect to access Cloud SQL from a production App Engine
>> instance, but if you want to deploy your schema or have any kind of access
>> to your Cloud SQL instance from your development workstation you'll need
>> the custom dialect.
>>
>>
>> If you use any extensions, you might run into this issue:
>> https://github.com/mitsuhiko/flask/issues/662
>>
>> I'm using a lot of extensions (Flask-SQLAlchemy, Flask-Admin,
>> Flask-Script, Flask-Security, Flask-WTF, Flask-Debugtoolbar, and more :) ),
>> and have worked around any issues that have come up.  Most of the
>> extensions can be fixed by changing how imports are done (as Flask-Cache
>> did - https://github.com/thadeusb/flask-cache/issues/38), although most
>> of the extensions just work (once you put them in your sys.path and in the
>> root of your project, or as I have, in a packages folder with the following
>> script: https://gist.github.com/4632192).
>>
>> Hopefully the Flask issue 662 is resolved (it seems to be an easy fix,
>> but I haven't had time to submit a pull request and write some tests to
>> verify).
>>
>> With Cloud SQL, I do most of my dev using a local sqlite and later MySQL
>> instance, and used the Flask/Werkzeug development server (and not the GAE
>> dev server as I tried to not have any dependences on google APIs (mail,
>> etc)).
>>
>> Let me know if I can help you out with any issues.  I've been planning to
>> create an extension to help with some GAE-specific items (flask-script
>> command to copy virtualenv packages to app's packages/ directory,
>> flask-script modified shell using the remote_api API to access the
>> Datastore remotely, etc).
>>
>> On Fri, Jan 25, 2013 at 12:36 AM, Paul Walsh <paulywalsh@gmail.com>wrote:
>>
>>> Does anyone have recent experience with Flask on GAE? I am about to
>>> start a project that must sit on GAE.
>>>
>>> We are undecided on using the non-relational data store, or using the
>>> new CloudSQL offering.
>>>
>>> Any advice, experience, tips based on recent deployments of Flask to GAE
>>> are appreciated, thanks.
>>>
>>>
>>> --
>>> *Paul Walsh*
>>> 0543551144
>>>
>>>
>>
>
> --
> *Paul Walsh*
> 0543551144
>
>

Re: [flask] Re: Flask on GAE in 2012/13

From:
Paul Walsh
Date:
2013-01-27 @ 12:27
Thanks guys, this is great help.

Also: This will be our first GAE project. I usually deploy myself to Linode
or AWS, so usually we are doing things like setting up Nginx, Gunicorn,
security on Ubuntu, etc.

We think GAE looks good, especially the progress they seem to have made
over the last year or so, and now with CloudSQL available.

However, some people still mention to us things about GAE being too
restrictive and so on.

How do you guys rate your experiences with the platform, especially, say,
compared to deploying yourself to something like AWS, or, using an
AWS-backed platform like Heroku?

Thanks.


*Paul Walsh*
0543551144


On 25 January 2013 15:51, Sean Lynch <techniq35@gmail.com> wrote:

> We chose Cloud SQL for our latest project for a few reasons:
>
>   - This project had a large reporting requirement, which would have been
> difficult without denormalizing our data a lot, or running a lot of map
> reduce jobs.  I considered using a hybrid approach of powering the app
> using the Datastore for speed / efficiency, but also writing to Cloud SQL
> for reporting (or to BigQuery, which would have been cheaper), but in the
> end, we decided it wasn't worth adding extra complexity and just use Cloud
> SQL completely.
>
>   - Along with reports, the app had a role-centric view that all data (and
> related data) visibility was based on what your role was (and also what
> associations you had). This greatly complicated how we would be able to
> store the data, especially with the restrictions on querying (
> 
https://developers.google.com/appengine/docs/python/datastore/queries#Restrictions_on_Queries
> )
>
>   - My team for this project didn't have any experience with the Datastore
> or map reduce (and I'm still working on getting a good grasp of the
> algorithms/techniques/best practices), so to keep me from having to do the
> lion share of the data acces / report work, I chose Cloud SQL.
>
> It's nice you get 1GB of free storage on Datastore, but as with my other
> projects, it's the number of operations you're typically paying the most
> for.  You have to spend a lot of time optimizing your indexes and
> extensively using memcache (which the newer `ndb` package helps a lot).
>
> Cloud SQL currently offers 1 free D0 instance till June, so it should give
> you enough time to try it out.  I wrote this app making sure I could host
> it elsewhere (AWS / Heroku) in case I ran into any issues, but so far I've
> been able to work through the (the SQLAlchemy dialect almost made me lose
> some sleep until I figured out what was going on).  Once you do setup a
> paid instance, it's nice you can setup a "dynamic" Per-Use instance like
> how App Engine instances work, where the instance shuts off and starts up
> automatically (after 15 minutes of usage).  You can also get a package
> plan, where you pay a fixed price per day, and the instances will stay up
> for 12 hours of inactivity.
>
>
> On Fri, Jan 25, 2013 at 2:12 AM, Paul Walsh <paulywalsh@gmail.com> wrote:
>
>> Thanks Sean.
>>
>> You've provided a wealth of info right here that puts me on good footing.
>>
>> Is there any significant reason you choose CloudSQL for your latest
>> project, and not the Datastore?
>>
>>
>> On Friday, January 25, 2013, Sean Lynch wrote:
>>
>>> I've deployed a few Flask-based apps on GAE, and finishing up my first
>>> CloudSQL backed one (I have 3 using the Datastore and 1 on Cloud SQL).
>>>
>>> If you plan to use Google Cloud SQL, I figure you'll use SQLAlchemy.  If
>>> so, you'll want to look at a few of my StackOverflow questions and answers:
>>>
>>> 
http://stackoverflow.com/questions/10763171/can-sqlalchemy-be-used-with-google-cloud-sql
>>>
>>> 
http://stackoverflow.com/questions/14224679/unable-to-connect-to-google-cloud-sql-from-development-server-using-sqlalchemy
>>>
>>> In short, SQLAlchemy supports Cloud SQL as of 0.7.8, but you'll want to
>>> use my custom dialect (and 0.8) until the changes are merged into a
>>> SQLAlchemy release. You can use the current (0.7.8 / 0.7.9)
>>> mysql+gaedbms:// dialect to access Cloud SQL from a production App Engine
>>> instance, but if you want to deploy your schema or have any kind of access
>>> to your Cloud SQL instance from your development workstation you'll need
>>> the custom dialect.
>>>
>>>
>>> If you use any extensions, you might run into this issue:
>>> https://github.com/mitsuhiko/flask/issues/662
>>>
>>> I'm using a lot of extensions (Flask-SQLAlchemy, Flask-Admin,
>>> Flask-Script, Flask-Security, Flask-WTF, Flask-Debugtoolbar, and more :) ),
>>> and have worked around any issues that have come up.  Most of the
>>> extensions can be fixed by changing how imports are done (as Flask-Cache
>>> did - https://github.com/thadeusb/flask-cache/issues/38), although most
>>> of the extensions just work (once you put them in your sys.path and in the
>>> root of your project, or as I have, in a packages folder with the following
>>> script: https://gist.github.com/4632192).
>>>
>>> Hopefully the Flask issue 662 is resolved (it seems to be an easy fix,
>>> but I haven't had time to submit a pull request and write some tests to
>>> verify).
>>>
>>> With Cloud SQL, I do most of my dev using a local sqlite and later MySQL
>>> instance, and used the Flask/Werkzeug development server (and not the GAE
>>> dev server as I tried to not have any dependences on google APIs (mail,
>>> etc)).
>>>
>>> Let me know if I can help you out with any issues.  I've been planning
>>> to create an extension to help with some GAE-specific items (flask-script
>>> command to copy virtualenv packages to app's packages/ directory,
>>> flask-script modified shell using the remote_api API to access the
>>> Datastore remotely, etc).
>>>
>>> On Fri, Jan 25, 2013 at 12:36 AM, Paul Walsh <paulywalsh@gmail.com>wrote:
>>>
>>>> Does anyone have recent experience with Flask on GAE? I am about to
>>>> start a project that must sit on GAE.
>>>>
>>>> We are undecided on using the non-relational data store, or using the
>>>> new CloudSQL offering.
>>>>
>>>> Any advice, experience, tips based on recent deployments of Flask to
>>>> GAE are appreciated, thanks.
>>>>
>>>>
>>>> --
>>>> *Paul Walsh*
>>>> 0543551144
>>>>
>>>>
>>>
>>
>> --
>> *Paul Walsh*
>> 0543551144
>>
>>
>

Re: [flask] Flask on GAE in 2012/13

From:
ian douglas
Date:
2013-01-25 @ 07:44
I've used this with enough success to keep using it for future projects:

http://github.com/kamalgill/flask-appengine-template
On Jan 24, 2013 10:24 PM, "Paul Walsh" <paulywalsh@gmail.com> wrote:

> Does anyone have recent experience with Flask on GAE? I am about to start
> a project that must sit on GAE.
>
> We are undecided on using the non-relational data store, or using the new
> CloudSQL offering.
>
> Any advice, experience, tips based on recent deployments of Flask to GAE
> are appreciated, thanks.
>
>
> --
> *Paul Walsh*
> 0543551144
>
>

Re: [flask] Flask on GAE in 2012/13

From:
Paul Walsh
Date:
2013-01-25 @ 09:58
Great, thanks. Looks like a good starting point.

*Paul Walsh*
0543551144


On 25 January 2013 09:44, ian douglas <ian.douglas@iandouglas.com> wrote:

> I've used this with enough success to keep using it for future projects:
>
> http://github.com/kamalgill/flask-appengine-template
> On Jan 24, 2013 10:24 PM, "Paul Walsh" <paulywalsh@gmail.com> wrote:
>
>> Does anyone have recent experience with Flask on GAE? I am about to start
>> a project that must sit on GAE.
>>
>> We are undecided on using the non-relational data store, or using the
>> new CloudSQL offering.
>>
>> Any advice, experience, tips based on recent deployments of Flask to GAE
>> are appreciated, thanks.
>>
>>
>> --
>> *Paul Walsh*
>> 0543551144
>>
>>