librelist archives

« back to archive

Blueprints and a Giant Site

Blueprints and a Giant Site

From:
Ethan Schlenker
Date:
2011-11-01 @ 19:48
Hello,

For work we are considering using Flask for a large and complex site 
(gamespot.com).  It seems that the suggested pattern is to break site into
different Blueprints based on functionality, eg: blueprints for the news 
area, one for the video area, one for the user area, one for the event 
area and so on.

But  we have at least a couple dozen areas with lots of logic in them and 
my concern is with having to load up all of those blueprints when the 
server starts.  It's potentially a lot of code, and they can't be 
dynamically loaded and unloaded.  Is there some cutoff where  it's 
overkill for Flask and perhaps we should use a different base framework?  

We've also talked about running the different areas as their own 
applications, but then we lose the ability to have everything under one 
roof (eg, can't use link_to(),...). And it's more of a monitoring concern,
and feels a little crazy.  But maybe it's not, and maybe I'm missing 
something.

So I'm curious if there's any wisdom around dealing with a situation 
similar to ours with Flask. Any suggestions or advice would be very 
welcome.

-Ethan

Re: [flask] Blueprints and a Giant Site

From:
Robert Mela
Date:
2011-11-04 @ 02:31
I don't see why the number of modules to load, and startup time, would be
much different for Flask would than for any other Python framework.   In
Django for example you would still be splitting the different areas into
their own Django views.

Python does offer one huge advantage at server start.   When a .py file is
first loaded the compiled code is stored in a .pyc file.  So the next time
the server starts it will not have to do the compilation step.

How frequently do you plan to be restarting servers?

Re: [flask] Blueprints and a Giant Site

From:
Ethan Schlenker
Date:
2011-11-04 @ 02:47
> How frequently do you plan to be restarting servers?

Hopefully not too often :-)  Build once a week on prod... So nothing too drastic.

I'm gonna take the general lack of concern over the number of Blueprints 
as a sign to not worry about it too much.

We'll see where we end up in a few months.  Working on putting a blog 
together, and will certainly post the link here when we get rolling.

Thanks,
Ethan

Re: [flask] Blueprints and a Giant Site

From:
Italo Maia
Date:
2012-02-14 @ 23:31
I would be very interested in that too! Have you guys tought about building
a prototype for testing purpose?

2011/11/3 Ethan Schlenker <ethan.schlenker@cbsinteractive.com>

> > How frequently do you plan to be restarting servers?
>
> Hopefully not too often :-)  Build once a week on prod... So nothing too
> drastic.
>
> I'm gonna take the general lack of concern over the number of Blueprints
> as a sign to not worry about it too much.
>
> We'll see where we end up in a few months.  Working on putting a blog
> together, and will certainly post the link here when we get rolling.
>
> Thanks,
> Ethan
>
>


-- 
"A arrogância é a arma dos fracos."

===========================
Italo Moreira Campelo Maia
Bacharel em Ciência da Computação - UECE
Desenvolvedor WEB e Desktop (Java, Python, Lua)
Coordenador do Pug-CE
-----------------------------------------------------
http://www.italomaia.com/
http://twitter.com/italomaia/
http://eusouolobomau.blogspot.com/
-----------------------------------------------------
Turtle Linux  9.10 - http://tiny.cc/blogturtle910
Turtle Linux 10.10 - http://bit.ly/cEw4ET
===========================

Re: [flask] Blueprints and a Giant Site

From:
bruce bushby
Date:
2012-05-02 @ 13:57
Add me to the list of those being very interested in tracking your
progress. Site look good, would luv to see the "Powered By Flask" on
it :)



On Tue, Feb 14, 2012 at 11:31 PM, Italo Maia <italo.maia@gmail.com> wrote:
> I would be very interested in that too! Have you guys tought about building
> a prototype for testing purpose?
>
>
> 2011/11/3 Ethan Schlenker <ethan.schlenker@cbsinteractive.com>
>>
>> > How frequently do you plan to be restarting servers?
>>
>> Hopefully not too often :-)  Build once a week on prod... So nothing too
>> drastic.
>>
>> I'm gonna take the general lack of concern over the number of Blueprints
>> as a sign to not worry about it too much.
>>
>> We'll see where we end up in a few months.  Working on putting a blog
>> together, and will certainly post the link here when we get rolling.
>>
>> Thanks,
>> Ethan
>>
>
>
>
> --
> "A arrogância é a arma dos fracos."
>
> ===========================
> Italo Moreira Campelo Maia
> Bacharel em Ciência da Computação - UECE
> Desenvolvedor WEB e Desktop (Java, Python, Lua)
> Coordenador do Pug-CE
> -----------------------------------------------------
> http://www.italomaia.com/
> http://twitter.com/italomaia/
> http://eusouolobomau.blogspot.com/
> -----------------------------------------------------
> Turtle Linux  9.10 - http://tiny.cc/blogturtle910
> Turtle Linux 10.10 - http://bit.ly/cEw4ET
> ===========================

Re: [flask] Blueprints and a Giant Site

From:
Simon Sapin
Date:
2011-11-01 @ 20:22
Le 01/11/2011 20:48, Ethan Schlenker a écrit :
> But  we have at least a couple dozen areas with lots of logic in them 
and my concern is with having to load up all of those blueprints when the 
server starts.  It's potentially a lot of code, and they can't be 
dynamically loaded and unloaded.  Is there some cutoff where  it's 
overkill for Flask and perhaps we should use a different base framework?

Hi,

Is your concern about how much time it takes to parse, compile and load 
all that Python code? It’s really faster than you think, especially with 
.pyc files. Also, depending on how you host your app in production, it 
only happens when you (re)start your server, not for every request.

I guess it would be possible to import stuff lazily (not until it is 
needed) but you shouldn’t worry about that until you have measurable 
performance problems.

You also talk of unloading. I guess it would be possible to unload 
Python modules. The only reason I can imagine is to save on RAM, but it 
really would not be much.

The issues you will more likely have with a lot of code are of 
maintainability. Blueprints are meant to help with that.

Regards,
-- 
Simon Sapin

Re: [flask] Blueprints and a Giant Site

From:
Robert Thurnher
Date:
2011-11-01 @ 22:04
Hi,

On 1 November 2011 21:22, Simon Sapin <simon.sapin@exyr.org> wrote:
> Le 01/11/2011 20:48, Ethan Schlenker a écrit :
>> But  we have at least a couple dozen areas with lots of logic in them 
and my concern is with having to load up all of those blueprints when the 
server starts.  It's potentially a lot of code, and they can't be 
dynamically loaded and unloaded.  Is there some cutoff where  it's 
overkill for Flask and perhaps we should use a different base framework?
>
> [...]
>
> The issues you will more likely have with a lot of code are of
> maintainability. Blueprints are meant to help with that.

As I'm currently in a somewhat similar situation evaluating to use
Flask for a rather large app relaunch ("large" mainly in terms of
existing domain data and expected site traffic, that is) I'd be
interested in reasons one would choose Flask over, say, Django.

The more tangible these reasons are, the better.

Regards,
-- Robert

Re: [flask] Blueprints and a Giant Site

From:
Ethan Schlenker
Date:
2011-11-02 @ 17:26
>> The issues you will more likely have with a lot of code are of
>> maintainability. Blueprints are meant to help with that.

I get that, but was wondering if there is any common wisdom along the 
lines of:  if you need more than N Blueprints, you're better off using Foo
framework / you should do it like this ... /  check out this large flask 
app using blueprints... / etc.  Any knowledge will be appreciated, though 
as Robert said:

> The more tangible these reasons are, the better.

I fully plan on sharing any lessons we learn from using Flask on a giant 
site, just need some help making sure we're headed in the right direction 
initially (as best we can tell)

-Ethan

Re: [flask] Blueprints and a Giant Site

From:
Joe Esposito
Date:
2011-11-02 @ 17:33
I'd be very interested in seeing this progress. Do you have a blog that
could record your progress with these Flask-related roadblocks and
successes of the site?

On Wed, Nov 2, 2011 at 1:26 PM, Ethan Schlenker <
ethan.schlenker@cbsinteractive.com> wrote:

> >> The issues you will more likely have with a lot of code are of
> >> maintainability. Blueprints are meant to help with that.
>
> I get that, but was wondering if there is any common wisdom along the
> lines of:  if you need more than N Blueprints, you're better off using Foo
> framework / you should do it like this ... /  check out this large flask
> app using blueprints... / etc.  Any knowledge will be appreciated, though
> as Robert said:
>
> > The more tangible these reasons are, the better.
>
> I fully plan on sharing any lessons we learn from using Flask on a giant
> site, just need some help making sure we're headed in the right direction
> initially (as best we can tell)
>
> -Ethan
>

Re: [flask] Blueprints and a Giant Site

From:
Zach Williams
Date:
2011-11-02 @ 17:51
I'd be interested in this, as well.


On Wed, Nov 2, 2011 at 10:33 AM, Joe Esposito <espo58@gmail.com> wrote:

> I'd be very interested in seeing this progress. Do you have a blog that
> could record your progress with these Flask-related roadblocks and
> successes of the site?
>
>
> On Wed, Nov 2, 2011 at 1:26 PM, Ethan Schlenker <
> ethan.schlenker@cbsinteractive.com> wrote:
>
>> >> The issues you will more likely have with a lot of code are of
>> >> maintainability. Blueprints are meant to help with that.
>>
>> I get that, but was wondering if there is any common wisdom along the
>> lines of:  if you need more than N Blueprints, you're better off using Foo
>> framework / you should do it like this ... /  check out this large flask
>> app using blueprints... / etc.  Any knowledge will be appreciated, though
>> as Robert said:
>>
>> > The more tangible these reasons are, the better.
>>
>> I fully plan on sharing any lessons we learn from using Flask on a giant
>> site, just need some help making sure we're headed in the right direction
>> initially (as best we can tell)
>>
>> -Ethan
>>
>
>