Having briefly looked over the docs, it appears one must load all view modules on each request. For large applications with dozens of handlers this could impact performance, especially if run as a cgi (ie. google app engine). Has anyone implemented a lazy-loading solution or have suggestions on how to implement one? Or am I, in fact, mistaken about the performance impact? -- John
Hi,
On 5/23/10 11:35 PM, John Kim wrote:
> especially if run as a cgi
Only if run as CGI. Otherwise modules stay loaded. Also on AppEngine
which looks like CGI but keeps the globals loaded.
Regards,
Armin
Using CGI is crazy, and App Engine supports WSGI: from google.appengine.ext.webapp import util from myapp import app util.run_wsgi_app(app) If you use WSGI your view modules should be loaded on bootup rather than on each request. mån 2010-05-24 klockan 06:35 +0900 skrev John Kim: > Having briefly looked over the docs, it appears one must load all view modules on each request. For large applications with dozens of handlers this could impact performance, especially if run as a cgi (ie. google app engine). Has anyone implemented a lazy-loading solution or have suggestions on how to implement one? Or am I, in fact, mistaken about the performance impact? > > -- John
Oh, I see. I was under the (wrong) impression that requests were sandboxed in app engine. So I suppose having 5 handlers or 500 handlers shouldn't make too much difference in regards to performance in app engine?
On Sun, May 23, 2010 at 6:56 PM, John Kim wrote: > Oh, I see. I was under the (wrong) impression that requests were sandboxed in app engine. > So I suppose having 5 handlers or 500 > handlers shouldn't make too much difference in regards > to performance in app engine? It should. Runtime instances are short-lived at this time and don't last more than a couple of minutes, and two requests never share the same runtime instance. They promise a (probably paid) feature for you to keep at least one instance "warm" and avoid the constant "cold start" (loading all modules and initializing stuff). This is not much a problem for very active apps afaik, but gets ugly for many people. Django apps can take seconds to respond to a request because of initialization, while "warm instances" have fast responses. This all means that to optimize Flask for App Engine you'd want to lazy load modules as much as you can, so that a "cold start" won't need to import all application modules and initialize stuff. So you'll maybe want to centralize routing and probably wrap views to be lazy loaded. For example, having as bootstrap: app = Flask(__name__) app.add_url_rule('/', 'home', view_func=LazyView('my_views.home_view')) LazyView would be something like: from werkzeug import import_string class LazyView(object): def __init__(self, name): self.name = name self.view = None def __call__(self, *args, **kwargs): if self.view is None: self.view = import_string(self.name) return self.view(*args, **kwargs) And so on. I haven't experimented much, but I hope this gives you an idea. -- rodrigo
Hi,
On 5/25/10 11:54 AM, Rodrigo Moraes wrote:
> LazyView would be something like:
I took that as an inspiration and drafted it into a pattern in the Flask
documentation: http://flask.pocoo.org/docs/patterns/lazyloading/
Regards,
Armin
Just a heads up for those of you using the new lazy-load view pattern: if
you plan to split up your views into multiple modules as I suspect most of
you will (otherwise I don't see the point in lazy-loading at all!) you may
run into problems if your modules contain views with clashing names ie.
front.home, admin.home
I overcame this problem by explicitly naming the endpoint when adding the
url rule:
app.add_url_rule('/', 'front.home',
view_func=LazyView('myapp.front.home'))
app.add_url_rule('/', 'admin.home',
view_func=LazyView('myapp.admin.home'))
Hi, On 5/27/10 8:36 AM, John Kim wrote: > Just a heads up for those of you using the new lazy-load view > pattern: if you plan to split up your views into multiple modules as > I suspect most of you will (otherwise I don't see the point in > lazy-loading at all!) you may run into problems if your modules > contain views with clashing names ie. front.home, admin.home The better way would be to use actual modules for that: app = Flask(__name__) from yourapplication.admin.urls import admin app.register_module(admin) And inside the admin's URL module you are using lazy views. That way you get the prefixing with the "admin." automatically. Regards, Armin
Also, wouldn't placing the routes in the urls module introduce circular dependencies with the app object? Or have I just misunderstood your approach? On May 27, 2010, at 7:39 PM, Armin Ronacher wrote: > The better way would be to use actual modules for that: > > app = Flask(__name__) > from yourapplication.admin.urls import admin > app.register_module(admin) > > And inside the admin's URL module you are using lazy views. That way > you get the prefixing with the "admin." automatically. > > > Regards, > Armin
Just to clarify, are you suggesting a structure like this? /myapp /__init__.py /admin /urls.py /views.py /front /urls.py /views.py On May 27, 2010, at 7:39 PM, Armin Ronacher wrote: > The better way would be to use actual modules for that: > > app = Flask(__name__) > from yourapplication.admin.urls import admin > app.register_module(admin) > > And inside the admin's URL module you are using lazy views. That way > you get the prefixing with the "admin." automatically. > > > Regards, > Armin
Hi,
On 5/28/10 12:04 AM, John Kim wrote:
> Just to clarify, are you suggesting a structure like this?
Jep.
Regards,
Armin
I ended up using a modified version of url
def url(rule, endpoint, strict_slashes = False, **kwargs):
if endpoint.startswith('.'):
import_name = 'aoide' + endpoint
endpoint = endpoint[1:]
else:
import_name = 'aoide.views.' + endpoint
view = LazyView(import_name)
app.add_url_rule(rule, endpoint, view,
strict_slashes=strict_slashes,
**kwargs)
But I always try to explicitly declare my endpoints. IE('front.home'
instead of just 'home' when in front module)
--
Thadeus
On Fri, May 28, 2010 at 10:57 AM, Armin Ronacher
<armin.ronacher@active-4.com> wrote:
> Hi,
>
> On 5/28/10 12:04 AM, John Kim wrote:
>> Just to clarify, are you suggesting a structure like this?
> Jep.
>
>
> Regards,
> Armin
>
What I ended up doing was just subclassing Module and overriding the
add_url_rule method:
class LazyModule(Module):
def __init__(self, import_name, name, url_prefix=None):
super(LazyModule, self).__init__(import_name, name, url_prefix)
self.import_name = import_name
def add_url_rule(self, rule, view_name, **options):
view = LazyView('.'.join((self.import_name, self.name, view_name)))
Module.add_url_rule(self, rule, view_name, view_func=view, **options)
Then defining the routes in /myapp/views/__init__.py file:
front = LazyModule(__name__, 'front')
front.add_url_rule('/', 'home')
front.add_url_rule('/login', 'login', methods=['GET', 'POST'])
admin = LazyModule(__name__, 'admin', '/admin')
admin.add_url_rule('/', 'home')
etc
Note that I had to explicitly set the module's name, otherwise flask would
prefix the endpoints with "views" which causes namespace conflicts. I
didn't really see the point of Modules at first, but now that I've
implemented them I'm starting to see the benefits, particularly with
Module-based pre/post processing. For example, I attached a login_required
decorator to all admin views with one before_request decorator:
@admin.before_request
@login_required
def admin_before_request():
pass
This is all just experimentation so I welcome anyone's suggestions.
On May 31, 2010, at 2:23 PM, Thadeus Burgess wrote:
> I ended up using a modified version of url
>
> def url(rule, endpoint, strict_slashes = False, **kwargs):
> if endpoint.startswith('.'):
> import_name = 'aoide' + endpoint
> endpoint = endpoint[1:]
> else:
> import_name = 'aoide.views.' + endpoint
>
> view = LazyView(import_name)
>
> app.add_url_rule(rule, endpoint, view,
> strict_slashes=strict_slashes,
> **kwargs)
>
> But I always try to explicitly declare my endpoints. IE('front.home'
> instead of just 'home' when in front module)
>
> --
> Thadeus
Thank you for this, Armin and Rodrigo. App engine deployment is now covered. I have to say, I appreciate Flask's approach: keeping the core light and focused and for slower people like me, documenting how to accomplish common tasks in the snippets and patterns pages. On May 26, 2010, at 10:00 PM, Armin Ronacher wrote: > Hi, > > On 5/25/10 11:54 AM, Rodrigo Moraes wrote: >> LazyView would be something like: > I took that as an inspiration and drafted it into a pattern in the Flask > documentation: http://flask.pocoo.org/docs/patterns/lazyloading/ > > > Regards, > Armin
Thanks Rodrigo. Your snippet works. I'm also investigating tipfy.^^ I don't want to turn this into an app engine/tipfy discussion, but as the author of tipfy I was wondering what you think about flask and using it with the app engine platform. I imagine the experience with flask and tipfy will be similar since they're built on the same stack but I was wondering if there are any compelling advantages to using a framework (like tipfy) built specifically for app engine. On May 25, 2010, at 6:54 PM, Rodrigo Moraes wrote: > > And so on. I haven't experimented much, but I hope this gives you an idea. > > -- rodrigo
On Tue, May 25, 2010 at 1:40 PM, John Kim wrote: > Thanks Rodrigo. Your snippet works. I'm also investigating tipfy.^^ > > I don't want to turn this into an app engine/tipfy discussion, but as the author of tipfy I was wondering what you think about flask and using it with the app engine platform. I imagine the experience with flask and tipfy will be similar since they're built on the same stack but I was wondering if there are any compelling advantages to using a framework (like tipfy) built specifically for app engine. Hey, You'll be fine with anything that fits your mind, I guess. Some stuff may require integration work, or you'd have to use webapp's default handlers (XMPP, deferred tasks, blobstore, inbound email). It should not be a lot of work to make them work in a Flasky way. Kay and Tipfy can be a reference for this. -- rodrigo
On 5/23/10 11:56 PM, John Kim wrote: > Oh, I see. I was under the (wrong) impression that requests were > sandboxed in app engine. So I suppose having 5 handlers or 500 > handlers shouldn't make too much difference in regards to performance > in app engine? Maybe on the first request, but not after that. That is an unfortunate side effect of decorators, but you can switch away from decorators and use a central URL map and lazy loading. I could probably add a snippet for that to the snippet website. Regards, Armin
Got it. I first got this idea for lazy-loading from Nick Johnson's blog here: http://blog.notdot.net/2010/02/Webapps-on-App-Engine-part-6-Lazy-loading. I suppose this approach doesn't apply to flask. A snippet would be useful. It seems at lot of flask users are interested in creating large applications. Personally speaking, I generally work on large application but prefer micro-frameworks because they're lighter, easier to understand at a glance and modify. Thanks for your assistance. On May 24, 2010, at 7:21 AM, Armin Ronacher wrote: > Maybe on the first request, but not after that. That is an unfortunate > side effect of decorators, but you can switch away from decorators and > use a central URL map and lazy loading. I could probably add a snippet > for that to the snippet website. > > > Regards, > Armin >
I really like flask because it gets out of my way and doesn't *force* anything on me in regards to application structure. A snippet would be helpful please Armin. I have not had time to look into this as of yet. -- Thadeus On Sun, May 23, 2010 at 11:14 PM, John Kim <koreansalaryman@gmail.com> wrote: > Got it. I first got this idea for lazy-loading from Nick Johnson's blog here: http://blog.notdot.net/2010/02/Webapps-on-App-Engine-part-6-Lazy-loading. I suppose this approach doesn't apply to flask. > > A snippet would be useful. It seems at lot of flask users are interested in creating large applications. Personally speaking, I generally work on large application but prefer micro-frameworks because they're lighter, easier to understand at a glance and modify. > > Thanks for your assistance. > > On May 24, 2010, at 7:21 AM, Armin Ronacher wrote: >> Maybe on the first request, but not after that. That is an unfortunate >> side effect of decorators, but you can switch away from decorators and >> use a central URL map and lazy loading. I could probably add a snippet >> for that to the snippet website. >> >> >> Regards, >> Armin >> > >