Locking datastore refactor
- From:
- Dag Sverre Seljebotn
- Date:
- 2011-03-28 @ 09:39
I've been working on a locking data store very slowly in spare moments.
I found that the only realistic option was to prepare for a simple
refactoring, because I could never write such code without very isolated
unit tests.
https://github.com/dagss/joblib/blob/locking/joblib/store.py
https://github.com/dagss/joblib/blob/locking/joblib/test/test_store.py
I haven't integrated it with Memory.py yet, but that should be a very
quick job. I know, waterfall development and all that, but I felt I
could justify it in this rather simple case.
Comments welcome on the data store API before I actually put the
necessary hour in to refactor Memory.py
Todo:
- Memory.clear() -> data store (which is still missing)
- Actually refactor Memory.py.
- Write stress-testing script to provoke deadlock situations and make
sure they're all catered for (they should be)
- Spawns thousands of processes with thousands of threads, which
gets SIGKILL-ed (or segfaults itself...) randomly, and also monkeypatch
"lockf" to make it randomly return a lock without locking :-)
Dag Sverre