librelist archives

« back to archive

serving static files

serving static files

From:
Carlos Gonzales
Date:
2012-08-27 @ 05:34
Hi. I've read the docs relating to serving static files and, while I
understand the best way to do this would be using a dedicated server, I've
really only seen examples on how to do it while in the development stage.

What are the advantages of using apache, for example, instead of letting
the wsgi object handle static files? How can I configure this behaviour?

I'm using apache with mod_wsgi.

Re: [flask] serving static files

From:
Mark Haase
Date:
2012-08-27 @ 14:30
The advantage is speed: I did some testing using blitz.io on my server (an
EC2 small instance) between using Flask/Werkzeug's send_file() and using
Apache to serve static assets. My test showed that the maximum number of
static files I could send per second from Flask was just under 60 (with
lots of timeouts in that configuration) and with Apache serving static
files the maximum hits per second is at least 228 (with no timeouts). The
blitz.io service I was using is capped at 250 simulated users unless you
have a paid account, so I am pretty sure that Apache could have scaled even
higher.

In your development environment, you do NOT want Apache serving static
files, in my opinion, because this makes your development environment more
complicated than it needs to be. Ideally you should be able to develop
against the Python server included with Flask, and only switch to Apache
when you are debugging problems specific to Apache. (You should of course
use Apache in your staging/test environments so that it is configured
identically to your production environment.)

If you want to set this up in Apache, there are a few ways to do it, but
the simplest is to use mod_alias:

    Alias /static /var/www/yourapp/path/to/static/

Put this Alias into your virtual host definition. (You need to change the
path, of course, to match wherever you are storing static files.)

On Mon, Aug 27, 2012 at 1:34 AM, Carlos Gonzales <carlos.rgn@gmail.com>wrote:

> Hi. I've read the docs relating to serving static files and, while I
> understand the best way to do this would be using a dedicated server, I've
> really only seen examples on how to do it while in the development stage.
>
> What are the advantages of using apache, for example, instead of letting
> the wsgi object handle static files? How can I configure this behaviour?
>
> I'm using apache with mod_wsgi.
>



-- 
Mark E. Haase

Re: [flask] serving static files

From:
Devon Stewart
Date:
2012-08-27 @ 18:43
Mark, I'm curious-- Did you have apache/nginx set up to handle X-SendFile,
or were you letting Flask/WSGI do the processing of the file?

I'm wondering if 60 vs 228 was because of Flask trying to process the 
files, or if it's really that much slower trying to match the route and 
find the file.

-Devon

On Aug 27, 2012, at 7:30, Mark Haase <mark.haase@lunarline.com> wrote:

> The advantage is speed: I did some testing using blitz.io on my server 
(an EC2 small instance) between using Flask/Werkzeug's send_file() and 
using Apache to serve static assets. My test showed that the maximum 
number of static files I could send per second from Flask was just under 
60 (with lots of timeouts in that configuration) and with Apache serving 
static files the maximum hits per second is at least 228 (with no 
timeouts). The blitz.io service I was using is capped at 250 simulated 
users unless you have a paid account, so I am pretty sure that Apache 
could have scaled even higher.
> 
> In your development environment, you do NOT want Apache serving static 
files, in my opinion, because this makes your development environment more
complicated than it needs to be. Ideally you should be able to develop 
against the Python server included with Flask, and only switch to Apache 
when you are debugging problems specific to Apache. (You should of course 
use Apache in your staging/test environments so that it is configured 
identically to your production environment.)
> 
> If you want to set this up in Apache, there are a few ways to do it, but
the simplest is to use mod_alias:
> 
>     Alias /static /var/www/yourapp/path/to/static/
> 
> Put this Alias into your virtual host definition. (You need to change 
the path, of course, to match wherever you are storing static files.)
> 
> On Mon, Aug 27, 2012 at 1:34 AM, Carlos Gonzales <carlos.rgn@gmail.com> wrote:
> Hi. I've read the docs relating to serving static files and, while I 
understand the best way to do this would be using a dedicated server, I've
really only seen examples on how to do it while in the development stage.
> 
> What are the advantages of using apache, for example, instead of letting
the wsgi object handle static files? How can I configure this behaviour?
> 
> I'm using apache with mod_wsgi.
> 
> 
> 
> -- 
> Mark E. Haase
> 

Re: [flask] serving static files

From:
Mark Haase
Date:
2012-08-27 @ 20:36
Devon,

That was purely a comparison between Flask/WSGI and Apache. I don't have
nginx running at all. So yes, you are correct, the slowdown is due to
running Python. I conducted the test because I was unsure how much of a
performance difference it would actually make, and I wanted to know that
before putting effort into a slightly more complex configuration than just
letting Flask handle everything.

Cheers,

On Mon, Aug 27, 2012 at 2:43 PM, Devon Stewart <blast@hardchee.se> wrote:

> Mark, I'm curious-- Did you have apache/nginx set up to handle X-SendFile,
> or were you letting Flask/WSGI do the processing of the file?
>
> I'm wondering if 60 vs 228 was because of Flask trying to process the
> files, or if it's really that much slower trying to match the route and
> find the file.
>
> -Devon
>
> On Aug 27, 2012, at 7:30, Mark Haase <mark.haase@lunarline.com> wrote:
>
> The advantage is speed: I did some testing using blitz.io on my server
> (an EC2 small instance) between using Flask/Werkzeug's send_file() and
> using Apache to serve static assets. My test showed that the maximum number
> of static files I could send per second from Flask was just under 60 (with
> lots of timeouts in that configuration) and with Apache serving static
> files the maxi mum hits per second is at least 228 (with no timeouts). The
> blitz.io service I was using is capped at 250 simulated users unless you
> have a paid account, so I am pretty sure that Apache could have scaled even
> higher.
>
>
> In your development environment, you do NOT want Apache serving static
> files, in my opinion, because this makes your development environment more
> complicated than it needs to be. Ideally you should be able to develop
> against the Python server included with Flask, and only switch to Apache
> when you are debugging problems specific to Apache. (You should of course
> use Apache in your staging/test environments so that it is configured
> identically to your production environment.)
>
> If you want to set this up in Apache, there are a few ways to do it, but
> the simplest is to use mod_alias:
>
>     Alias /static /var/www/yourapp/path/to/static/
>
> Put this Alias into your virtual host definition. (You need to change the
> path, of course, to match wherever you are storing static files.)
>
> On Mon, Aug 27, 2012 at 1:34 AM, Carlos Gonzales <carlos.rgn@gmail.com>wrote:
>
>> Hi. I've read the docs relating to serving static files and, while I
>> understand the best way to do this would be using a dedicated server, I've
>> really only seen examples on how to do it while in the development stage.
>>
>> What are the advantages of using apache, for example, instead of letting
>> the wsgi object handle static files? How can I configure this behaviour?
>>
>> I'm using apache with mod_wsgi.
>>
>
>
>
> --
> Mark E. Haase
>
>


-- 
Mark E. Haase
CISSP
Sr. Security Software Engineer
www.lunarline.com
3300 N Fairfax Drive, Suite 308, Arlington, VA 22201
571-334-8408

"Solutions Built on Security" TM
Lunarline, Inc. is an ISO 9001 and CMMI Level 2 Certified SDVOSB
Information Assurance\ Cyber Security Services Company.

Re: [flask] serving static files

From:
Carlos Gonzales
Date:
2012-08-28 @ 07:30
Thanks for all your input, guys. Mark: you were very helpful in regards to
my second question. I think another benefit of not using send_file is
having more flexibility with cache headers. I've noticed that even though
flask usually responds with a max-age of 12 hours, more often than not,
before those 12 hours are up, I'll load up a page and around half of the
static files will get fetched from the server even if these haven't been
modified. (Might've imagined this.) Plus I'm not sure I particularly like
setting Last-Modified. If I understand this correctly, this header is
making browsers still communicate with the server for each static file.
Seems like such a waste when the response is always 304 Not Modified.

Anyways, a general file structure I'm using is the following:

app/
  __init__.py
  blueprints/
    blueprint1/
      __init__.py
      static/
        css/
          index.css
        js/
          index.js
      templates/
        blueprint1/
          index.html
  app.py

I run my application on a domain's root so in order to serve the static
files I suppose I should set my document root to the blueprints directory.
Although according to the docs this wouldn't do much good and I'd actually
have to create an alias of some sort. I'm not quite sure about this part.

Saludos,
Carlos R. Gonzales


On Mon, Aug 27, 2012 at 2:36 PM, Mark Haase <mark.haase@lunarline.com>wrote:

> Devon,
>
> That was purely a comparison between Flask/WSGI and Apache. I don't have
> nginx running at all. So yes, you are correct, the slowdown is due to
> running Python. I conducted the test because I was unsure how much of a
> performance difference it would actually make, and I wanted to know that
> before putting effort into a slightly more complex configuration than just
> letting Flask handle everything.
>
> Cheers,
>
> On Mon, Aug 27, 2012 at 2:43 PM, Devon Stewart <blast@hardchee.se> wrote:
>
>> Mark, I'm curious-- Did you have apache/nginx set up to handle
>> X-SendFile, or were you letting Flask/WSGI do the processing of the file?
>>
>> I'm wondering if 60 vs 228 was because of Flask trying to process the
>> files, or if it's really that much slower trying to match the route and
>> find the file.
>>
>> -Devon
>>
>> On Aug 27, 2012, at 7:30, Mark Haase <mark.haase@lunarline.com> wrote:
>>
>> The advantage is speed: I did some testing using blitz.io on my server
>> (an EC2 small instance) between using Flask/Werkzeug's send_file() and
>> using Apache to serve static assets. My test showed that the maximum number
>> of static files I could send per second from Flask was just under 60 (with
>> lots of timeouts in that configuration) and with Apache serving static
>> files the maxi mum hits per second is at least 228 (with no timeouts). The
>> blitz.io service I was using is capped at 250 simulated users unless you
>> have a paid account, so I am pretty sure that Apache could have scaled even
>> higher.
>>
>>
>> In your development environment, you do NOT want Apache serving static
>> files, in my opinion, because this makes your development environment more
>> complicated than it needs to be. Ideally you should be able to develop
>> against the Python server included with Flask, and only switch to Apache
>> when you are debugging problems specific to Apache. (You should of course
>> use Apache in your staging/test environments so that it is configured
>> identically to your production environment.)
>>
>> If you want to set this up in Apache, there are a few ways to do it, but
>> the simplest is to use mod_alias:
>>
>>     Alias /static /var/www/yourapp/path/to/static/
>>
>> Put this Alias into your virtual host definition. (You need to change the
>> path, of course, to match wherever you are storing static files.)
>>
>> On Mon, Aug 27, 2012 at 1:34 AM, Carlos Gonzales <carlos.rgn@gmail.com>wrote:
>>
>>> Hi. I've read the docs relating to serving static files and, while I
>>> understand the best way to do this would be using a dedicated server, I've
>>> really only seen examples on how to do it while in the development stage.
>>>
>>> What are the advantages of using apache, for example, instead of letting
>>> the wsgi object handle static files? How can I configure this behaviour?
>>>
>>> I'm using apache with mod_wsgi.
>>>
>>
>>
>>
>> --
>> Mark E. Haase
>>
>>
>
>
> --
> Mark E. Haase
> CISSP
> Sr. Security Software Engineer
> www.lunarline.com
> 3300 N Fairfax Drive, Suite 308, Arlington, VA 22201
> 571-334-8408
>
> "Solutions Built on Security" TM
> Lunarline, Inc. is an ISO 9001 and CMMI Level 2 Certified SDVOSB
> Information Assurance\ Cyber Security Services Company.
>
>

Re: [flask] serving static files

From:
Carlos Gonzales
Date:
2012-08-29 @ 10:25
I ended up doing the following in case anyone finds it useful:

<VirtualHost *:80>
  ServerName ddg.local

  AliasMatch ^/static/(.*) /usr/local/src/ddg/ddg/static/$1
  AliasMatch (?i)^/([a-z/_]+)/static/(.*)
/usr/local/src/ddg/ddg/blueprints/$1/static/$2

  ExpiresActive On
  <FilesMatch (?i)\.(css|js)$>
    ExpiresDefault "access plus 1 hour"
  </FilesMatch>
  <FilesMatch (?i)\.(gif|jpe?g|png)$>
    ExpiresDefault "access plus 1 week"
  </FilesMatch>
  <FilesMatch (?i)jquery>
    ExpiresDefault "access plus 1 year"
  </FilesMatch>

  WSGIDaemonProcess ddg user=ddg threads=100
  WSGIScriptAlias / /usr/local/src/ddg/ddg.wsgi
  <Directory /usr/local/src/ddg>
    WSGIProcessGroup ddg
    WSGIApplicationGroup %{GLOBAL}
    Order deny,allow
    Allow from all
  </Directory>
</VirtualHost>

Saludos,
Carlos R. Gonzales


On Tue, Aug 28, 2012 at 1:30 AM, Carlos Gonzales <carlos.rgn@gmail.com>wrote:

> Thanks for all your input, guys. Mark: you were very helpful in regards to
> my second question. I think another benefit of not using send_file is
> having more flexibility with cache headers. I've noticed that even though
> flask usually responds with a max-age of 12 hours, more often than not,
> before those 12 hours are up, I'll load up a page and around half of the
> static files will get fetched from the server even if these haven't been
> modified. (Might've imagined this.) Plus I'm not sure I particularly like
> setting Last-Modified. If I understand this correctly, this header is
> making browsers still communicate with the server for each static file.
> Seems like such a waste when the response is always 304 Not Modified.
>
> Anyways, a general file structure I'm using is the following:
>
> app/
>   __init__.py
>   blueprints/
>     blueprint1/
>       __init__.py
>       static/
>         css/
>           index.css
>         js/
>           index.js
>       templates/
>         blueprint1/
>           index.html
>   app.py
>
> I run my application on a domain's root so in order to serve the static
> files I suppose I should set my document root to the blueprints directory.
> Although according to the docs this wouldn't do much good and I'd actually
> have to create an alias of some sort. I'm not quite sure about this part.
>
> Saludos,
> Carlos R. Gonzales
>
>
>
> On Mon, Aug 27, 2012 at 2:36 PM, Mark Haase <mark.haase@lunarline.com>wrote:
>
>> Devon,
>>
>> That was purely a comparison between Flask/WSGI and Apache. I don't have
>> nginx running at all. So yes, you are correct, the slowdown is due to
>> running Python. I conducted the test because I was unsure how much of a
>> performance difference it would actually make, and I wanted to know that
>> before putting effort into a slightly more complex configuration than just
>> letting Flask handle everything.
>>
>> Cheers,
>>
>> On Mon, Aug 27, 2012 at 2:43 PM, Devon Stewart <blast@hardchee.se> wrote:
>>
>>> Mark, I'm curious-- Did you have apache/nginx set up to handle
>>> X-SendFile, or were you letting Flask/WSGI do the processing of the file?
>>>
>>> I'm wondering if 60 vs 228 was because of Flask trying to process the
>>> files, or if it's really that much slower trying to match the route and
>>> find the file.
>>>
>>> -Devon
>>>
>>> On Aug 27, 2012, at 7:30, Mark Haase <mark.haase@lunarline.com> wrote:
>>>
>>> The advantage is speed: I did some testing using blitz.io on my server
>>> (an EC2 small instance) between using Flask/Werkzeug's send_file() and
>>> using Apache to serve static assets. My test showed that the maximum number
>>> of static files I could send per second from Flask was just under 60 (with
>>> lots of timeouts in that configuration) and with Apache serving static
>>> files the maxi mum hits per second is at least 228 (with no timeouts). The
>>> blitz.io service I was using is capped at 250 simulated users unless
>>> you have a paid account, so I am pretty sure that Apache could have scaled
>>> even higher.
>>>
>>>
>>> In your development environment, you do NOT want Apache serving static
>>> files, in my opinion, because this makes your development environment more
>>> complicated than it needs to be. Ideally you should be able to develop
>>> against the Python server included with Flask, and only switch to Apache
>>> when you are debugging problems specific to Apache. (You should of course
>>> use Apache in your staging/test environments so that it is configured
>>> identically to your production environment.)
>>>
>>> If you want to set this up in Apache, there are a few ways to do it, but
>>> the simplest is to use mod_alias:
>>>
>>>     Alias /static /var/www/yourapp/path/to/static/
>>>
>>> Put this Alias into your virtual host definition. (You need to change
>>> the path, of course, to match wherever you are storing static files.)
>>>
>>> On Mon, Aug 27, 2012 at 1:34 AM, Carlos Gonzales <carlos.rgn@gmail.com>wrote:
>>>
>>>> Hi. I've read the docs relating to serving static files and, while I
>>>> understand the best way to do this would be using a dedicated server, I've
>>>> really only seen examples on how to do it while in the development stage.
>>>>
>>>> What are the advantages of using apache, for example, instead of
>>>> letting the wsgi object handle static files? How can I configure this
>>>> behaviour?
>>>>
>>>> I'm using apache with mod_wsgi.
>>>>
>>>
>>>
>>>
>>> --
>>> Mark E. Haase
>>>
>>>
>>
>>
>> --
>> Mark E. Haase
>> CISSP
>> Sr. Security Software Engineer
>> www.lunarline.com
>> 3300 N Fairfax Drive, Suite 308, Arlington, VA 22201
>> 571-334-8408
>>
>> "Solutions Built on Security" TM
>> Lunarline, Inc. is an ISO 9001 and CMMI Level 2 Certified SDVOSB
>> Information Assurance\ Cyber Security Services Company.
>>
>>
>

Re: [flask] serving static files

From:
Andres Rand
Date:
2012-08-27 @ 07:17
Hi.

I've done this with nginx and uwisgi. I configured nginx to serve
static files at certain url (e.g. example.com/app/static/). With
apache you sould be able to do it similary. You can redefine flasks
default static url location like this and then serve it on however you
coose:
app = Flask(__name__, static_url_path='<insert_url_path_here>')
url_for('static', filename='file.txt') # returns url for static file,
url redefined at app creation

--
Andres Rand

2012/8/27 Carlos Gonzales <carlos.rgn@gmail.com>:
> Hi. I've read the docs relating to serving static files and, while I
> understand the best way to do this would be using a dedicated server, I've
> really only seen examples on how to do it while in the development stage.
>
> What are the advantages of using apache, for example, instead of letting the
> wsgi object handle static files? How can I configure this behaviour?
>
> I'm using apache with mod_wsgi.

Re: [flask] serving static files

From:
Devon Stewart
Date:
2012-08-27 @ 07:59
I'm assuming you mean send_file(), if not, my apologies.

The advantage of using send_file() is that instead of Python having to 
read the whole file and tell the web server about it, the web server just 
goes to the file itself and reads/sends the file without the extra 
overhead (and possibly better optimized as well, since apache is handling 
the connection at a low level).

send_file can be used anywhere (not just /static/); Also, I suspect (can't
check right now) /static/ uses send_file internally.

-Devon

On Aug 27, 2012, at 0:17, Andres Rand <hrrand@gmail.com> wrote:

> Hi.
> 
> I've done this with nginx and uwisgi. I configured nginx to serve
> static files at certain url (e.g. example.com/app/static/). With
> apache you sould be able to do it similary. You can redefine flasks
> default static url location like this and then serve it on however you
> coose:
> app = Flask(__name__, static_url_path='<insert_url_path_here>')
> url_for('static', filename='file.txt') # returns url for static file,
> url redefined at app creation
> 
> --
> Andres Rand
> 
> 2012/8/27 Carlos Gonzales <carlos.rgn@gmail.com>:
>> Hi. I've read the docs relating to serving static files and, while I
>> understand the best way to do this would be using a dedicated server, I've
>> really only seen examples on how to do it while in the development stage.
>> 
>> What are the advantages of using apache, for example, instead of letting the
>> wsgi object handle static files? How can I configure this behaviour?
>> 
>> I'm using apache with mod_wsgi.

Re: [flask] serving static files

From:
Kirill Klenov
Date:
2012-08-27 @ 09:49
I serve static files with nginx+uwsgi. My server dirs structure like this:

../project
|__ deploy (config files)
|__ logs (project logs)
|__ source (source code app)
|__ static (static files here)

Nginx config like this:

...
location ^~ /static/ { root .../project; expires max }
location / { uwsgi_pass project.uwsgi }
...


Also, for easy collect static files I writed this module:
https://github.com/klen/Flask-Collect
For now I just run: "./manage.py collect" (in my case this run
automatic on server update) and get static files from blueprints in
static folder.


Kirill Klenov
web developer
______________________________________________________________________
horneds@gmail.com | http://klen.github.com
mobile: +7 906 772-36-20


2012/8/27 Devon Stewart <blast@hardchee.se>:
> I'm assuming you mean send_file(), if not, my apologies.
>
> The advantage of using send_file() is that instead of Python having to 
read the whole file and tell the web server about it, the web server just 
goes to the file itself and reads/sends the file without the extra 
overhead (and possibly better optimized as well, since apache is handling 
the connection at a low level).
>
> send_file can be used anywhere (not just /static/); Also, I suspect 
(can't check right now) /static/ uses send_file internally.
>
> -Devon
>
> On Aug 27, 2012, at 0:17, Andres Rand <hrrand@gmail.com> wrote:
>
>> Hi.
>>
>> I've done this with nginx and uwisgi. I configured nginx to serve
>> static files at certain url (e.g. example.com/app/static/). With
>> apache you sould be able to do it similary. You can redefine flasks
>> default static url location like this and then serve it on however you
>> coose:
>> app = Flask(__name__, static_url_path='<insert_url_path_here>')
>> url_for('static', filename='file.txt') # returns url for static file,
>> url redefined at app creation
>>
>> --
>> Andres Rand
>>
>> 2012/8/27 Carlos Gonzales <carlos.rgn@gmail.com>:
>>> Hi. I've read the docs relating to serving static files and, while I
>>> understand the best way to do this would be using a dedicated server, I've
>>> really only seen examples on how to do it while in the development stage.
>>>
>>> What are the advantages of using apache, for example, instead of letting the
>>> wsgi object handle static files? How can I configure this behaviour?
>>>
>>> I'm using apache with mod_wsgi.