[Scons-users] Parallel MD5
Eric van Gyzen
eric at vangyzen.net
Wed Feb 26 11:55:44 EST 2014
On Feb 24, 2014, at 6:33 PM, Dirk Bächle wrote:
> On 22.02.2014 14:12, Eric van Gyzen wrote:
>> SCons Users:
>> It seems that SCons uses a single thread to calculate the MD5 sums of the sources, regardless of the -jN flag. Is this correct? When working in large projects with thousands of files, and only modifying one file, parallelizing the MD5 calculations across the -jN CPUs could be a major performance improvement.
> I've done some work on analysing and profiling SCons performance in the
> past (see http://www.scons.org/wiki/WhySconsIsNotSlow ), and so far I
> didn't find that MD5 computation is a real bottleneck.
> If you've experienced major slowdowns due to the calculation of
> hashsums, I'd be very interested in your results. This includes not only
> the actual figures, but also information about the used project setup
> and the evaluation process.
I am embarrassed to admit that my claim was based on a hunch, with no scientific experimentation of any kind. It is complete conjecture. Thank you for your performance analysis. It is very well done. Obviously, I need to do a similar analysis with our code base.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Scons-users