[Scons-users] Need help to improve the incremental build time while using SCons

[Bill Deegan]

I added an improvement which yielded 15% speedup on a large codebase (6000+
c++ files).
A couple micro optimizations in some inner code which gets hit a lot.
(replaced about 30 lines of code)
It's in the 3.0.0 beta.

I'm working on performance currently. (As well as getting the 3.0.0 release
out)
We have some good real world projects profiling information which is
pointing towards some hot spots.

Please give 3.0.0alpha a try.
It's available on pypitest via:

pip install -i https://testpypi.python.org/pypi scons

Also take a look at the following wiki pages:
https://bitbucket.org/scons/scons/wiki/GoFastButton
https://bitbucket.org/scons/scons/wiki/WhySconsIsNotSlow

To some extent SCons will always be slower than Make as it is doing more to
ensure that files are rebuilt when needed.
(Make doesn't check if command lines for compilation changes, or if the
compiler changes, and many other things which should cause a rebuild).

I'm hoping to bring the delta down a bit.
-Bill



On Tue, Jun 20, 2017 at 7:06 AM, William Blevins <wblevins001 at gmail.com>
wrote:

> The 2.5 series was about cross-language build support. 2.5.1 performance
> improvements were just to address a 2.5.0 inefficiency.
>
> On Jun 20, 2017 4:59 AM, "Thomas Berg" <merlin66b at gmail.com> wrote:
>
> On Tue, Jun 20, 2017 at 7:29 AM, Basil Mathew <iambasilmathew at gmail.com>
> wrote:
>
>> 1. Is the nearly 6 fold increase in the incremental build time as
>> compared to GMake on expected lines? I did go through the archive e-mails,
>> but could not get a concrete answer to the question.
>>
>
> Yes, this is my experience with large C++ projects too. With ~9000 .cpp
> files, we get a no-op build time of about 5 minutes after implementing many
> of the optimisations from the mailing list.
>
>
>
>> 2. Are there any other options that would allow me to get a more
>> favorable incremental build time?
>>
>
> In our build, tests have shown that we could cut off minutes by not
> scanning third party headers. But we update our libraries sufficiently
> often that it's not something we can use by default.
>
> SCons is also known to spend a lot of time doing string substitutions,
> generating compiler commands. You may get some speedup from modifying the
> compiler command, hard coding as much as possible in the command itself
> instead of adding things to CCFLAGS and CPPDEFINES. You can do this for the
> parts that are the same for your entire build.
>
> The biggest win for incremental builds is to support partial builds: in
> our build we detect it if a developer launches scons in a subdirectory, and
> in this case we execute the SConscripts for this subtree only, meaning we
> don't inform scons about the rest of the build targets. You may need to
> modify your SConscripts and even implement workarounds for scons bugs (in
> some cases) to support this. And of course developers then have to know
> what they are doing to get a correct build - they have to know that their
> modification only requires building a particular subtree. But we get 10
> second incremental build times from this, for single .cpp file changes (and
> headers too, assuming they are only included in the subtree in question),
> so for us this is an absolute requirement.
>
> Cheers,
> Thomas Berg
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20170620/fdfdba42/attachment-0001.html>