[Scons-users] Performance of version 2.5.0 vs 2.3.0 on Windows host dropped significantly
William Blevins
wblevins001 at gmail.com
Tue Aug 9 10:21:24 EDT 2016
Henrik,
Unfortunately, the 0.2s difference for 2.3.6 cannot be helped. The 2.5.0
change didn't affect much code, but greatly affected implicit dependency
scanning for the better. I don't know if it will affect you personally, but
2.5.0 added cross-language implicit scanning support.
V/R,
William
On Tue, Aug 9, 2016 at 1:45 PM, <hmnews at proconx.com> wrote:
> Hi to all who responded,
>
> William's pull request reduces the total build time from 12s down to 3.5s!
> So this would solve the issue.
>
> I compared some Scons versions for our reference project (C++, 161 C/C++
> files eg targets) which is a real-world gcc cross-compilation project not
> some odd test case.
>
> 2.3.0: 2.4s
> 2.3.5: 3.15s
> 2.3.6: 3.3s
> 2.5.0: 12s
> 2.5.0 with pull request applied: 3.5s
>
> Based on that, 2.3.0 is still the unbeaten winner, 2.3.5 and 2.3.6 I can
> live with and once the pull request is applied to 2.5 build times are at
> least again within a reasonable range for the project size.
>
> And yes, the build time scales with the number of object files in the
> project.
>
> I can't test on Linux, my build environment lives purely on Windows.
>
> Thanks for all the help. Scons is a very useful tool.
>
> Henrik
>
>
> William Blevins wrote:
>
>> Hi,
>>
>> This may be a known issue. SCons 2.5.0 changed how implicit dependencies
>> were generated which added cross-language support (and fixed potentially
>> missing implicit dependencies). As Alexey Klimkin has already pointed
>> out, the change could be optimized safely by caching the path_func
>> results. This wasn't something the original Executor code required,
>> since scanner_key(s) never changed after previously picking the initial
>> scanner. Please see
>> https://bitbucket.org/scons/scons/pull-requests/345/optimize
>> -implicit-dependency-scan/diff
>>
>> Your performance issue is most likely related. My fault entirely. I
>> hadn't expected those calls to be that expensive or perhaps I just
>> didn't have a test build big enough to notice.
>>
>> If you have the ability to try a patched version of SCons from the PR
>> above, then feedback would be appreciated.
>>
>> V/R,
>> William
>>
>> On Mon, Aug 8, 2016 at 9:37 AM, Vasily <just.one.man at yandex.ru
>> <mailto:just.one.man at yandex.ru>> wrote:
>>
>> Hi,
>>
>> You might also do some profiling of fast and slow cases and compare
>> the results to see if there is any hint on what's slowing down your
>> build.
>>
>> Thanks,
>> Vasily
>>
>> 08 авг. 2016 г. 10:17 пользователь "Dirk Bächle" <tshortik at gmx.de
>> <mailto:tshortik at gmx.de>> написал:
>>
>> Hi Henrik,
>>
>> On 08.08.2016 03:23, hmnews at proconx.com
>> <mailto:hmnews at proconx.com> wrote:
>>
>> Hi all,
>>
>> We just updated to Scons 2.5.0 after using happily 2.3.0 for
>> years.
>>
>> Sadly we found performance to have dropped significantly and
>> I wonder
>> what the contributing factor could be.
>>
>> Platform: Windows 10
>> Python: 2.7.12 64-bit
>>
>>
>> some more questions about this: Can you compare the times for a
>> build under Linux as well, or do you build under Windows only?
>>
>> [...]
>>
>> However with 2.5.0 execution time blows out to 11.8s, more
>> than 4 times
>> longer!!!
>>
>>
>> You say that this is a benchmark project. What happens to the
>> "real" projects? Do they scale, i.e. do they also take about 4
>> times longer each?
>>
>> [...]
>>
>> I appears that Scons takes much longer in 2.5.0 to analyse
>> the build
>> targets. This is also apparent when running scons-2.5.0
>> --debug=prepare
>> which takes noticeable longer to to print the file list.
>>
>> I also compared the dependency tree for both version using
>> --debug=tree
>> and they are identical.
>>
>>
>> Good to know, because we have changed "scanning" in a few
>> places, which could've explained the longer times too.
>>
>> Has anybody experienced a similar behaviour and is there any
>> tuning
>> parameter to make 2.5.0 offer the same performance of 2.3.0?
>>
>>
>> If there were such a parameter, it would definitely be "on" by
>> default. ;)
>>
>>
>> Is there any debug option to analyse why Scons 2.5.0 takes
>> such a long
>> execution time?
>>
>>
>> In general, SCons uses most of its time during a "zero build"
>> (nothing to do), "subst"ing variable values. If you're prepared
>> for a long and technical read have a look at
>> https://bitbucket.org/scons/scons/wiki/WhySconsIsNotSlow
>> <https://bitbucket.org/scons/scons/wiki/WhySconsIsNotSlow> . You
>> might also want to check out the "fastcpp" tool at
>> https://bitbucket.org/dirkbaechle/scons_fastcpp
>> <https://bitbucket.org/dirkbaechle/scons_fastcpp> to get an idea
>> of how to "tweak" your environment/toolchain to get more speed
>> in general.
>>
>> I hope these first comments already help you a little further.
>>
>> Best regards,
>>
>> Dirk
>>
>>
>>
>> Thanks
>>
>> Henrik
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org <mailto:Scons-users at scons.org>
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>> <https://pairlist4.pair.net/mailman/listinfo/scons-users>
>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org <mailto:Scons-users at scons.org>
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>> <https://pairlist4.pair.net/mailman/listinfo/scons-users>
>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org <mailto:Scons-users at scons.org>
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>> <https://pairlist4.pair.net/mailman/listinfo/scons-users>
>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20160809/cb7d87ca/attachment-0001.html>
More information about the Scons-users
mailing list