Logging to file from workers
- Matthias Seise
- 2014-04-04 @ 07:46
Dear joblist users,
I'm trying to work out how to (correctly) log from multiple processes
(using joblib.Parallel) to a single file. How do you handle it?
(I'm working in Windows 7, python 2.7.2 64 bit) - The "standard" way using
a network logger as in the logging Cookbook doesn't work, the network
logger crashes after a while (~1hour).
Any pointers would be appreciated!
Re: [joblib] Logging to file from workers
- Olivier Grisel
- 2014-04-04 @ 08:34
I would make each worker process write results in its own file
(possibly with timestamp information) and from time to time have a
single finalizer process that
consolidate the results in the final file.