librelist archives

« back to archive

Compiler thread safety

Compiler thread safety

From:
Paolo Redaelli
Date:
2011-09-09 @ 08:59
I've been reading a post of Wouter van Oortmerssen on Google+ (
https://plus.google.com/101155429896002364181/posts I'm not sure if it is
public) regarding a new desktop machine optimized for compiling speed. He
were considering either an Intel i7 (6 cores, 12 threads) or a dual Xean (12
cores, 24 thread). He concluded that compiling scales well with the number
of cores with the i7 being faster in most benchmarks except complete rebuild
of "huge" codebases and that linking is mostly IO bound.
We were thinking to use separate processes and ZeroMQ for messaging.
So looking at the sheer number of threads that current high-end CPUs are
able to run concurrently and seeing that they are all able to run 2 thread
per core I think that we shall at least reconsider the possibility to make
the compiler thread safe.
So far we know for sure that there are some static variables in the run
time; so far I found that

   - in deep_twin.c there is the hash table "se_deep_twin_memory" and the
   counter "se_deep_equal_start_counter". I think that using INTERNALs we may
   reimplement this in pure Eiffel
   - in base.c the runtime handler se_runtime_handler_t
   - the Garbage Collector is mostly non thread-safe; I haven't yet studied
   it, so I cannot say anything except that we may - at least temporary -use
   Boehm's conservative garbage collector (package libgc-dev in Ubuntu) which
   has been usable in SmartEiffel as far as I remember and it is thread safe.
   Not an optimal solution but feasible for a while
   - profile and sedb would require extensive changes

I started re-implementing deep_twin in my fork; I hope to have something
testable in a week or two.

Compiler thread safety

From:
Paolo Redaelli
Date:
2011-09-13 @ 21:44
I've been reading a post of Wouter van Oortmerssen on Google+ (
https://plus.google.com/101155429896002364181/posts
<https://plus.google.com/101155429896002364181/posts> I'm not sure if it
is public) regarding a new desktop machine optimized for compiling
speed. He were considering either an Intel i7 (6 cores, 12 threads) or a
dual Xean (12 cores, 24 thread). He concluded that compiling scales well
with the number of cores with the i7 being faster in most benchmarks
except complete rebuild of "huge" codebases and that linking is mostly
IO bound. 
We were thinking to use separate processes and ZeroMQ for messaging.
So looking at the sheer number of threads that current high-end CPUs are
able to run concurrently and seeing that they are all able to run 2
thread per core I think that we shall at least reconsider the
possibility to make the compiler thread safe.
So far we know for sure that there are some static variables in the run
time; so far I found that

    * in deep_twin.c there is the hash table "se_deep_twin_memory" and
      the counter "se_deep_equal_start_counter". I think that using
      INTERNALs we may reimplement this in pure Eiffel
    * in base.c the runtime handler se_runtime_handler_t
    * the Garbage Collector is mostly non thread-safe; I haven't yet
      studied it, so I cannot say anything except that we may - at least
      temporary -use Boehm's conservative garbage collector (package
      libgc-dev in Ubuntu) which has been usable in SmartEiffel as far
      as I remember and it is thread safe. Not an optimal solution but
      feasible for a while
    * profile and sedb would require extensive changes

I started re-implementing deep_twin in my fork; I hope to have something
testable in a week or two.

PS: I reposted it after having problems with mail. I subscribed from two
different addresses to see if it does actually work