Hello, First of all: thanks a ton for writing joblib, it has helped us tremendously! I'm trying to get joblib's Memory.cache() working on some (already) decorated functions and methods. For some reason, when I use *args and **kwargs, the cache doesn't work. See below for details, sorry I haven't had time to dig deeper (but I guess this has to do with the known limitation briefly mentioned at http://packages.python.org/joblib/memory.html#gotchas i.e. "memory cannot be used on some complex objects, eg a callable object with a __call__ method") Thank you. Best, -- Nicolas Pinto, PhD http://web.mit.edu/pinto # to reproduce the bug from joblib import Memory mem = Memory(cachedir='./bug', verbose=False) mem.clear() import numpy as np def add4(func): def _add4(*args, **kwargs): val = args[0] return func(val+4) return _add4 def add4_no_args(func): def _add4_no_args(val): return func(val+4) return _add4_no_args square = np.square add4_square = add4(np.square) add4_no_args_square = add4_no_args(np.square) print "Ground truth:" for val in 1,2: print square(val), add4_square(val), add4_no_args_square(val) print "with joblib's Memory.cache():" square = mem.cache(np.square) add4_square = mem.cache(add4(np.square)) add4_no_args_square = mem.cache(add4_no_args(np.square)) for val in 1,2: print square(val), add4_square(val), add4_no_args_square(val) # output: Ground truth: 1 25 25 4 36 36 with joblib's Memory.cache(): 1 25 25 4 25 36
Hey Nicolas, I have gotten your mail, but since then things have been hectic and I haven't had even time to read it carefully. I think I'll look at it in the train, going off to the Alps for the week end. I might not be able to report back until Monday. Sorry for the delay Gael On Thu, Mar 17, 2011 at 09:48:19PM +0000, Nicolas Pinto wrote: > Hello, > First of all: thanks a ton for writing joblib, it has helped us tremendously! > I'm trying to get joblib's Memory.cache() working on some (already) > decorated functions and methods. For some reason, when I use *args and > **kwargs, the cache doesn't work. See below for details, sorry I > haven't had time to dig deeper (but I guess this has to do with the > known limitation briefly mentioned at > http://packages.python.org/joblib/memory.html#gotchas i.e. "memory > cannot be used on some complex objects, eg a callable object with a > __call__ method") > Thank you. > Best,
Ok, should be fixed in branch 0.5.X on github. Can you please confirm. Gael ----- Original message ----- > Hey Nicolas, > > I have gotten your mail, but since then things have been hectic and I > haven't had even time to read it carefully. I think I'll look at it in > the train, going off to the Alps for the week end. I might not be able to > report back until Monday. > > Sorry for the delay > > Gael > > On Thu, Mar 17, 2011 at 09:48:19PM +0000, Nicolas Pinto wrote: > > Hello, > > > First of all: thanks a ton for writing joblib, it has helped us > > tremendously! > > > I'm trying to get joblib's Memory.cache() working on some (already) > > decorated functions and methods. For some reason, when I use *args and > > **kwargs, the cache doesn't work. See below for details, sorry I > > haven't had time to dig deeper (but I guess this has to do with the > > known limitation briefly mentioned at > > http://packages.python.org/joblib/memory.html#gotchas i.e. "memory > > cannot be used on some complex objects, eg a callable object with a > > __call__ method") > > > Thank you.
Fabulous! It works now. Thanks a ton for your help! N On Sat, Mar 19, 2011 at 2:47 PM, Gael Varoquaux <gael.varoquaux@normalesup.org> wrote: > Ok, should be fixed in branch 0.5.X on github. Can you please confirm. > > Gael > > ----- Original message ----- >> Hey Nicolas, >> >> I have gotten your mail, but since then things have been hectic and I >> haven't had even time to read it carefully. I think I'll look at it in >> the train, going off to the Alps for the week end. I might not be able to >> report back until Monday. >> >> Sorry for the delay >> >> Gael >> >> On Thu, Mar 17, 2011 at 09:48:19PM +0000, Nicolas Pinto wrote: >> > Hello, >> >> > First of all: thanks a ton for writing joblib, it has helped us >> > tremendously! >> >> > I'm trying to get joblib's Memory.cache() working on some (already) >> > decorated functions and methods. For some reason, when I use *args and >> > **kwargs, the cache doesn't work. See below for details, sorry I >> > haven't had time to dig deeper (but I guess this has to do with the >> > known limitation briefly mentioned at >> > http://packages.python.org/joblib/memory.html#gotchas i.e. "memory >> > cannot be used on some complex objects, eg a callable object with a >> > __call__ method") >> >> > Thank you. > > -- Nicolas Pinto, PhD http://web.mit.edu/pinto
On Fri, Mar 18, 2011 at 02:16:45PM +0100, Gael Varoquaux wrote: > I have gotten your mail, but since then things have been hectic and I > haven't had even time to read it carefully. I think I'll look at it in > the train, going off to the Alps for the week end. I might not be able to > report back until Monday. I have had a look in the train. It is indeed a bug. The simplest test case is:: from nose.tools import assert_equal from joblib.func_inspect import filter_args def f(*args, **kwargs): return None assert_equal(filter_args(f, [], 1), {'*': [1], '**': {}}) I am working on it, but I am not supposed to work this week end, ... So I am picking up my computer when nobody is looking. G
Thanks a lot Gael for the blazing fast answer. Let me know if there is anything I can help you with. Best, N On Sat, Mar 19, 2011 at 7:39 AM, Gael Varoquaux <gael.varoquaux@normalesup.org> wrote: > On Fri, Mar 18, 2011 at 02:16:45PM +0100, Gael Varoquaux wrote: >> I have gotten your mail, but since then things have been hectic and I >> haven't had even time to read it carefully. I think I'll look at it in >> the train, going off to the Alps for the week end. I might not be able to >> report back until Monday. > > I have had a look in the train. It is indeed a bug. The simplest test > case is:: > > from nose.tools import assert_equal > > from joblib.func_inspect import filter_args > > def f(*args, **kwargs): > return None > > assert_equal(filter_args(f, [], 1), {'*': [1], '**': {}}) > > I am working on it, but I am not supposed to work this week end, ... So I > am picking up my computer when nobody is looking. > > G > -- Nicolas Pinto, PhD http://web.mit.edu/pinto