[Scons-users] Need help to improve the incremental build time while using SCons

[Dirk Bächle]

Hi Basil,


I'd like to second Bill here


On 20.06.2017 17:02, Bill Deegan wrote:
>
> [...]
>
>
>
> To some extent SCons will always be slower than Make as it is doing 
> more to ensure that files are rebuilt when needed.
> (Make doesn't check if command lines for compilation changes, or if 
> the compiler changes, and many other things which should cause a rebuild).
>
>
. I've done quite some profiling of the SCons core in the past (see 
https://bitbucket.org/scons/scons/wiki/WhySconsIsNotSlow ), and it's not 
possible to find a single spot for an "update run" where all the runtime 
is spent. Then it would be easy to make things faster with little effort.
Instead the runtime is spread over numerous methods and routines. On 
closer inspection, all this "overhead" is needed because it offers the 
kind of flexibility and extensibility that SCons is really about.
One could shortcut certain operations, but not without "crippling" the 
core's functionality in more or less drastic ways.

Having said this, as a first step you may want to call SCons with the 
"--debug=time" option. This will give you an impression about how much 
time is consumed for reading the SConscripts (not a lot of speedup 
possible here, unless something is really, really wrong with how you 
setup your build) vs the actual "check-whether-everything-is-up-to-date" 
time.
A large part of the latter will be spent on substituting string values 
and computing build signatures. You could try to make things faster by 
using the "fastcpp" Tool that I wrote (see 
https://bitbucket.org/scons/scons/wiki/ToolsIndex or 
https://bitbucket.org/dirkbaechle/scons_fastcpp directly). It will 
pre-expand certain string variables and this can save up to 20% update 
time. It also disables a default feature of SCons that tries to find the 
used compiler/tools executables in the $PATH and caches their MD5 hash 
as well.
Depending on your $PATH setting, this can already help a lot.

Using a different Decider, as being proposed very often, won't help at 
all. SCons internally *always* computes and updates the MD5 hashes 
too...such that no complete rebuild is required when switching from one 
Decider type to another on two consecutive builds.

For even faster edit-compile cycles you should try to avoid a simple 
"scons", which tries to build everything. If you simply know that your 
latest changes have been to the "foomagic" program (or a library that's 
required), a "scons foomagic" will give you a drastic speedup. This is 
because now SCons will only expand and trace back the dependencies for 
the target (folder) you specified.
This works even if the required dependencies for "foomagic" aren't in 
the same folder (or below it in the directory hierarchy).

Finally, try the "interactive" mode, which let's you avoid the cost of 
re-reading all the SConscripts on every build/update. Combined with 
specifying a concrete target (folder), I was able to run a "no changes 
update" for a library of 1000 files in a 100k file project in about 1-2 
seconds.

I hope this all gives you some further ideas.

Best regards,

Dirk

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20170620/3a0162f7/attachment.html>