librelist archives

« back to archive

HTTP caching library for wsgi/Flask

HTTP caching library for wsgi/Flask

From:
Yaniv Aknin
Date:
2013-02-04 @ 09:51
Hi,

I'm looking for a library that will add caching to an existing Flask app
(either as a Flask extension or as wsgi middleware, I prefer the former,
but I prefer even more a mature and good package).

I've seen quite a few caching related extensions/apps as well as werkzeug's
caching support, but I was rather surprised that all I found was
application-level caching (i.e., I decide which functions to memoize,
get/set cache values, etc) whereas I'm looking for HTTP level caching
(controlled by Cache-Control, ETags, Expires, etc).

In other words, I'm looking for a drop in into my stack that will do what
Varnish or even django.middlewaer.cache would do for me.

Does this exist and am I just blind?

Cheers,
 - Yaniv

Re: [flask] HTTP caching library for wsgi/Flask

From:
dorian i
Date:
2013-02-05 @ 12:11
So you want something that caches the whole generated response in
redis/memcache/whatever?

On Mon, Feb 4, 2013 at 10:51 AM, Yaniv Aknin <yaniv@aknin.name> wrote:

> Hi,
>
> I'm looking for a library that will add caching to an existing Flask app
> (either as a Flask extension or as wsgi middleware, I prefer the former,
> but I prefer even more a mature and good package).
>
> I've seen quite a few caching related extensions/apps as well as
> werkzeug's caching support, but I was rather surprised that all I found was
> application-level caching (i.e., I decide which functions to memoize,
> get/set cache values, etc) whereas I'm looking for HTTP level caching
> (controlled by Cache-Control, ETags, Expires, etc).
>
> In other words, I'm looking for a drop in into my stack that will do what
> Varnish or even django.middlewaer.cache would do for me.
>
> Does this exist and am I just blind?
>
> Cheers,
>  - Yaniv
>

Re: [flask] HTTP caching library for wsgi/Flask

From:
Yaniv Aknin
Date:
2013-02-05 @ 12:25
Yes.


On 5 February 2013 14:11, dorian i <dorian.hoxha@gmail.com> wrote:

> So you want something that caches the whole generated response in
> redis/memcache/whatever?
>
>
> On Mon, Feb 4, 2013 at 10:51 AM, Yaniv Aknin <yaniv@aknin.name> wrote:
>
>> Hi,
>>
>> I'm looking for a library that will add caching to an existing Flask app
>> (either as a Flask extension or as wsgi middleware, I prefer the former,
>> but I prefer even more a mature and good package).
>>
>> I've seen quite a few caching related extensions/apps as well as
>> werkzeug's caching support, but I was rather surprised that all I found was
>> application-level caching (i.e., I decide which functions to memoize,
>> get/set cache values, etc) whereas I'm looking for HTTP level caching
>> (controlled by Cache-Control, ETags, Expires, etc).
>>
>> In other words, I'm looking for a drop in into my stack that will do what
>> Varnish or even django.middlewaer.cache would do for me.
>>
>> Does this exist and am I just blind?
>>
>> Cheers,
>>  - Yaniv
>>
>
>

Re: [flask] HTTP caching library for wsgi/Flask

From:
Khalil El Kouhen
Date:
2013-02-05 @ 13:34
Is this what you are looking for ?

http://werkzeug.pocoo.org/docs/contrib/cache/

Le mardi 5 février 2013, Yaniv Aknin a écrit :

> Yes.
>
>
> On 5 February 2013 14:11, dorian i 
<dorian.hoxha@gmail.com<javascript:_e({}, 'cvml', 
'dorian.hoxha@gmail.com');>
> > wrote:
>
>> So you want something that caches the whole generated response in
>> redis/memcache/whatever?
>>
>>
>> On Mon, Feb 4, 2013 at 10:51 AM, Yaniv Aknin 
<yaniv@aknin.name<javascript:_e({}, 'cvml', 'yaniv@aknin.name');>
>> > wrote:
>>
>>> Hi,
>>>
>>> I'm looking for a library that will add caching to an existing Flask app
>>> (either as a Flask extension or as wsgi middleware, I prefer the former,
>>> but I prefer even more a mature and good package).
>>>
>>> I've seen quite a few caching related extensions/apps as well as
>>> werkzeug's caching support, but I was rather surprised that all I found was
>>> application-level caching (i.e., I decide which functions to memoize,
>>> get/set cache values, etc) whereas I'm looking for HTTP level caching
>>> (controlled by Cache-Control, ETags, Expires, etc).
>>>
>>> In other words, I'm looking for a drop in into my stack that will do
>>> what Varnish or even django.middlewaer.cache would do for me.
>>>
>>> Does this exist and am I just blind?
>>>
>>> Cheers,
>>>  - Yaniv
>>>
>>
>>
>

-- 
Khalil EL KOUHEN
Directeur HEC Rabat
www.hec.ma  Tél.: 212 (0)5 37 67 12 76 Fax.: 212 (0)5 37 67 12 77

Re: [flask] HTTP caching library for wsgi/Flask

From:
Yaniv Aknin
Date:
2013-02-05 @ 13:54
No.

Unless I'm horribly mistaken, werkzeug.contrib.cache is a generic caching
engine with pluggable backends.

I'm looking for something more high level (very possibly built on top of
werkzeug.contrib.cache) which looks at all HTTP requests coming to my app
and all HTTP responses sent from my app (that thing can be either a Flask
extension or a WSGI middleware, I don't want an external caching reverse
proxy like Varnish).

When a response is sent from my app, that something should look at its HTTP
based caching headers (like Cache-Control, Expires, ETag, etc) and decide
if this response can be cached and for how long. When a request arrives at
my app, that something should look at the request (along with cache related
headers) and decide if it can serve that request from my cache, bypassing
my app altogether.

Ideally, that something would also have convenient decorators I could put
on my views to add these HTTP cache headers to my responses.

I'm beginning to believe something like this doesn't exist neither for
Flask nor for WSGI in general, and to be frank, I'm quite surprised.

 - Yaniv


On 5 February 2013 15:34, Khalil El Kouhen <khalil.elkouhen@gmail.com>wrote:

> Is this what you are looking for ?
>
> http://werkzeug.pocoo.org/docs/contrib/cache/
>
> Le mardi 5 février 2013, Yaniv Aknin a écrit :
>
> Yes.
>>
>>
>> On 5 February 2013 14:11, dorian i <dorian.hoxha@gmail.com> wrote:
>>
>>> So you want something that caches the whole generated response in
>>> redis/memcache/whatever?
>>>
>>>
>>> On Mon, Feb 4, 2013 at 10:51 AM, Yaniv Aknin <yaniv@aknin.name> wrote:
>>>
>>>> Hi,
>>>>
>>>> I'm looking for a library that will add caching to an existing Flask
>>>> app (either as a Flask extension or as wsgi middleware, I prefer the
>>>> former, but I prefer even more a mature and good package).
>>>>
>>>> I've seen quite a few caching related extensions/apps as well as
>>>> werkzeug's caching support, but I was rather surprised that all I found was
>>>> application-level caching (i.e., I decide which functions to memoize,
>>>> get/set cache values, etc) whereas I'm looking for HTTP level caching
>>>> (controlled by Cache-Control, ETags, Expires, etc).
>>>>
>>>> In other words, I'm looking for a drop in into my stack that will do
>>>> what Varnish or even django.middlewaer.cache would do for me.
>>>>
>>>> Does this exist and am I just blind?
>>>>
>>>> Cheers,
>>>>  - Yaniv
>>>>
>>>
>>>
>>
>
> --
> Khalil EL KOUHEN
> Directeur HEC Rabat
> www.hec.ma  Tél.: 212 (0)5 37 67 12 76 Fax.: 212 (0)5 37 67 12 77
>
>

Re: [flask] HTTP caching library for wsgi/Flask

From:
Sean Lynch
Date:
2013-02-05 @ 13:46
Flask-Cache (an extension built upon werkzeug cache)

http://packages.python.org/Flask-Cache/


On Tue, Feb 5, 2013 at 8:34 AM, Khalil El Kouhen
<khalil.elkouhen@gmail.com>wrote:

> Is this what you are looking for ?
>
> http://werkzeug.pocoo.org/docs/contrib/cache/
>
> Le mardi 5 février 2013, Yaniv Aknin a écrit :
>
> Yes.
>>
>>
>> On 5 February 2013 14:11, dorian i <dorian.hoxha@gmail.com> wrote:
>>
>>> So you want something that caches the whole generated response in
>>> redis/memcache/whatever?
>>>
>>>
>>> On Mon, Feb 4, 2013 at 10:51 AM, Yaniv Aknin <yaniv@aknin.name> wrote:
>>>
>>>> Hi,
>>>>
>>>> I'm looking for a library that will add caching to an existing Flask
>>>> app (either as a Flask extension or as wsgi middleware, I prefer the
>>>> former, but I prefer even more a mature and good package).
>>>>
>>>> I've seen quite a few caching related extensions/apps as well as
>>>> werkzeug's caching support, but I was rather surprised that all I found was
>>>> application-level caching (i.e., I decide which functions to memoize,
>>>> get/set cache values, etc) whereas I'm looking for HTTP level caching
>>>> (controlled by Cache-Control, ETags, Expires, etc).
>>>>
>>>> In other words, I'm looking for a drop in into my stack that will do
>>>> what Varnish or even django.middlewaer.cache would do for me.
>>>>
>>>> Does this exist and am I just blind?
>>>>
>>>> Cheers,
>>>>  - Yaniv
>>>>
>>>
>>>
>>
>
> --
> Khalil EL KOUHEN
> Directeur HEC Rabat
> www.hec.ma  Tél.: 212 (0)5 37 67 12 76 Fax.: 212 (0)5 37 67 12 77
>
>

Re: [flask] HTTP caching library for wsgi/Flask

From:
Yaniv Aknin
Date:
2013-02-05 @ 14:02
As far as I can tell, Flask-Cache doesn't consider HTTP headers. I want
something that works at the HTTP level, honours headers like Vary or
If-None-Match, etc.

Again, maybe I'm missing something, but I doubt you can deal with HTTP
caching without having the word 'control' more in your codebase.

$ git grep -i control
flask_cache/__init__.py:    This class is used to control the cache
objects.
$

I want to use HTTP caching because HTTP caches can exist in many different
places - the user-agent, intermediary gateways and server side (commonly
called a caching reverse proxy). This piece of code I'm looking for will do
the role of a caching reverse proxy, but I want to set my HTTP caching
logic once and have every compliant entity along the way honour it.

Other ideas?

 - Yaniv

On 5 February 2013 15:46, Sean Lynch <techniq35@gmail.com> wrote:

> Flask-Cache (an extension built upon werkzeug cache)
>
> http://packages.python.org/Flask-Cache/
>
>
> On Tue, Feb 5, 2013 at 8:34 AM, Khalil El Kouhen <
> khalil.elkouhen@gmail.com> wrote:
>
>> Is this what you are looking for ?
>>
>> http://werkzeug.pocoo.org/docs/contrib/cache/
>>
>> Le mardi 5 février 2013, Yaniv Aknin a écrit :
>>
>> Yes.
>>>
>>>
>>> On 5 February 2013 14:11, dorian i <dorian.hoxha@gmail.com> wrote:
>>>
>>>> So you want something that caches the whole generated response in
>>>> redis/memcache/whatever?
>>>>
>>>>
>>>> On Mon, Feb 4, 2013 at 10:51 AM, Yaniv Aknin <yaniv@aknin.name> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I'm looking for a library that will add caching to an existing Flask
>>>>> app (either as a Flask extension or as wsgi middleware, I prefer the
>>>>> former, but I prefer even more a mature and good package).
>>>>>
>>>>> I've seen quite a few caching related extensions/apps as well as
>>>>> werkzeug's caching support, but I was rather surprised that all I found was
>>>>> application-level caching (i.e., I decide which functions to memoize,
>>>>> get/set cache values, etc) whereas I'm looking for HTTP level caching
>>>>> (controlled by Cache-Control, ETags, Expires, etc).
>>>>>
>>>>> In other words, I'm looking for a drop in into my stack that will do
>>>>> what Varnish or even django.middlewaer.cache would do for me.
>>>>>
>>>>> Does this exist and am I just blind?
>>>>>
>>>>> Cheers,
>>>>>  - Yaniv
>>>>>
>>>>
>>>>
>>>
>>
>> --
>> Khalil EL KOUHEN
>> Directeur HEC Rabat
>> www.hec.ma  Tél.: 212 (0)5 37 67 12 76 Fax.: 212 (0)5 37 67 12 77
>>
>>
>

Re: [flask] HTTP caching library for wsgi/Flask

From:
Steven Kryskalla
Date:
2013-02-05 @ 15:37
On Tue, Feb 5, 2013 at 6:02 AM, Yaniv Aknin <yaniv@aknin.name> wrote:
> I want to use HTTP caching because HTTP caches can exist in many different
> places - the user-agent, intermediary gateways and server side (commonly
> called a caching reverse proxy). This piece of code I'm looking for will do
> the role of a caching reverse proxy, but I want to set my HTTP caching logic
> once and have every compliant entity along the way honour it.
>
> Other ideas?

Here are some WSGI middleware libraries that implement some of the
things you mention (etags, cache-control):

https://github.com/repoze/repoze.accelerator
http://pypi.python.org/pypi/wsgistate/
http://pypi.python.org/pypi/wsgi_cache
http://www.ollycope.com/software/pesto/caching.html

I haven't used any of them but they may be worth a try.

-Steve

Re: [flask] HTTP caching library for wsgi/Flask

From:
Yaniv Aknin
Date:
2013-02-05 @ 15:55
I found wsgistate and wsgi_cache, but not the other two. I'll look into
them, accelerator looks interesting.

Thanks!
 - Yaniv


On 5 February 2013 17:37, Steven Kryskalla <skryskalla@gmail.com> wrote:

> On Tue, Feb 5, 2013 at 6:02 AM, Yaniv Aknin <yaniv@aknin.name> wrote:
> > I want to use HTTP caching because HTTP caches can exist in many
> different
> > places - the user-agent, intermediary gateways and server side (commonly
> > called a caching reverse proxy). This piece of code I'm looking for will
> do
> > the role of a caching reverse proxy, but I want to set my HTTP caching
> logic
> > once and have every compliant entity along the way honour it.
> >
> > Other ideas?
>
> Here are some WSGI middleware libraries that implement some of the
> things you mention (etags, cache-control):
>
> https://github.com/repoze/repoze.accelerator
> http://pypi.python.org/pypi/wsgistate/
> http://pypi.python.org/pypi/wsgi_cache
> http://www.ollycope.com/software/pesto/caching.html
>
> I haven't used any of them but they may be worth a try.
>
> -Steve
>

Re: [flask] HTTP caching library for wsgi/Flask

From:
dorian i
Date:
2013-02-05 @ 15:14
Flask Cache accepts a callable called unless and maybe there you can
implement your logic?

On Tue, Feb 5, 2013 at 3:02 PM, Yaniv Aknin <yaniv@aknin.name> wrote:

> As far as I can tell, Flask-Cache doesn't consider HTTP headers. I want
> something that works at the HTTP level, honours headers like Vary or
> If-None-Match, etc.
>
> Again, maybe I'm missing something, but I doubt you can deal with HTTP
> caching without having the word 'control' more in your codebase.
>
> $ git grep -i control
> flask_cache/__init__.py:    This class is used to control the cache
> objects.
> $
>
> I want to use HTTP caching because HTTP caches can exist in many different
> places - the user-agent, intermediary gateways and server side (commonly
> called a caching reverse proxy). This piece of code I'm looking for will do
> the role of a caching reverse proxy, but I want to set my HTTP caching
> logic once and have every compliant entity along the way honour it.
>
> Other ideas?
>
>  - Yaniv
>
>
> On 5 February 2013 15:46, Sean Lynch <techniq35@gmail.com> wrote:
>
>> Flask-Cache (an extension built upon werkzeug cache)
>>
>> http://packages.python.org/Flask-Cache/
>>
>>
>> On Tue, Feb 5, 2013 at 8:34 AM, Khalil El Kouhen <
>> khalil.elkouhen@gmail.com> wrote:
>>
>>> Is this what you are looking for ?
>>>
>>> http://werkzeug.pocoo.org/docs/contrib/cache/
>>>
>>> Le mardi 5 février 2013, Yaniv Aknin a écrit :
>>>
>>> Yes.
>>>>
>>>>
>>>> On 5 February 2013 14:11, dorian i <dorian.hoxha@gmail.com> wrote:
>>>>
>>>>> So you want something that caches the whole generated response in
>>>>> redis/memcache/whatever?
>>>>>
>>>>>
>>>>> On Mon, Feb 4, 2013 at 10:51 AM, Yaniv Aknin <yaniv@aknin.name> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I'm looking for a library that will add caching to an existing Flask
>>>>>> app (either as a Flask extension or as wsgi middleware, I prefer the
>>>>>> former, but I prefer even more a mature and good package).
>>>>>>
>>>>>> I've seen quite a few caching related extensions/apps as well as
>>>>>> werkzeug's caching support, but I was rather surprised that all I found was
>>>>>> application-level caching (i.e., I decide which functions to memoize,
>>>>>> get/set cache values, etc) whereas I'm looking for HTTP level caching
>>>>>> (controlled by Cache-Control, ETags, Expires, etc).
>>>>>>
>>>>>> In other words, I'm looking for a drop in into my stack that will do
>>>>>> what Varnish or even django.middlewaer.cache would do for me.
>>>>>>
>>>>>> Does this exist and am I just blind?
>>>>>>
>>>>>> Cheers,
>>>>>>  - Yaniv
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>> --
>>> Khalil EL KOUHEN
>>> Directeur HEC Rabat
>>> www.hec.ma  Tél.: 212 (0)5 37 67 12 76 Fax.: 212 (0)5 37 67 12 77
>>>
>>>
>>
>

Re: [flask] HTTP caching library for wsgi/Flask

From:
Yaniv Aknin
Date:
2013-02-05 @ 15:38
It's possible, but the whole reason I asked is that I was hoping someone
else already implemented that logic.

See, it's not trivial, nor is it "mine" - hence it's called HTTP caching
(see rfc2616 <http://www.ietf.org/rfc/rfc2616.txt> and
this<http://www.mnot.net/cache_docs/>very readable explanation)...

Anyway, cheers,
 - Yaniv


On 5 February 2013 17:14, dorian i <dorian.hoxha@gmail.com> wrote:

> Flask Cache accepts a callable called unless and maybe there you can
> implement your logic?
>
>
> On Tue, Feb 5, 2013 at 3:02 PM, Yaniv Aknin <yaniv@aknin.name> wrote:
>
>> As far as I can tell, Flask-Cache doesn't consider HTTP headers. I want
>> something that works at the HTTP level, honours headers like Vary or
>> If-None-Match, etc.
>>
>> Again, maybe I'm missing something, but I doubt you can deal with HTTP
>> caching without having the word 'control' more in your codebase.
>>
>> $ git grep -i control
>> flask_cache/__init__.py:    This class is used to control the cache
>> objects.
>> $
>>
>> I want to use HTTP caching because HTTP caches can exist in many
>> different places - the user-agent, intermediary gateways and server side
>> (commonly called a caching reverse proxy). This piece of code I'm looking
>> for will do the role of a caching reverse proxy, but I want to set my HTTP
>> caching logic once and have every compliant entity along the way honour it.
>>
>> Other ideas?
>>
>>  - Yaniv
>>
>>
>> On 5 February 2013 15:46, Sean Lynch <techniq35@gmail.com> wrote:
>>
>>> Flask-Cache (an extension built upon werkzeug cache)
>>>
>>> http://packages.python.org/Flask-Cache/
>>>
>>>
>>> On Tue, Feb 5, 2013 at 8:34 AM, Khalil El Kouhen <
>>> khalil.elkouhen@gmail.com> wrote:
>>>
>>>> Is this what you are looking for ?
>>>>
>>>> http://werkzeug.pocoo.org/docs/contrib/cache/
>>>>
>>>> Le mardi 5 février 2013, Yaniv Aknin a écrit :
>>>>
>>>> Yes.
>>>>>
>>>>>
>>>>> On 5 February 2013 14:11, dorian i <dorian.hoxha@gmail.com> wrote:
>>>>>
>>>>>> So you want something that caches the whole generated response in
>>>>>> redis/memcache/whatever?
>>>>>>
>>>>>>
>>>>>> On Mon, Feb 4, 2013 at 10:51 AM, Yaniv Aknin <yaniv@aknin.name>wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm looking for a library that will add caching to an existing Flask
>>>>>>> app (either as a Flask extension or as wsgi middleware, I prefer the
>>>>>>> former, but I prefer even more a mature and good package).
>>>>>>>
>>>>>>> I've seen quite a few caching related extensions/apps as well as
>>>>>>> werkzeug's caching support, but I was rather surprised that all I 
found was
>>>>>>> application-level caching (i.e., I decide which functions to memoize,
>>>>>>> get/set cache values, etc) whereas I'm looking for HTTP level caching
>>>>>>> (controlled by Cache-Control, ETags, Expires, etc).
>>>>>>>
>>>>>>> In other words, I'm looking for a drop in into my stack that will do
>>>>>>> what Varnish or even django.middlewaer.cache would do for me.
>>>>>>>
>>>>>>> Does this exist and am I just blind?
>>>>>>>
>>>>>>> Cheers,
>>>>>>>  - Yaniv
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>> --
>>>> Khalil EL KOUHEN
>>>> Directeur HEC Rabat
>>>> www.hec.ma  Tél.: 212 (0)5 37 67 12 76 Fax.: 212 (0)5 37 67 12 77
>>>>
>>>>
>>>
>>
>