[Scons-users] Scons always rebuilding

[Bill Deegan]

Stijn,

Having experience with LSF, in general your approach is not going to
improve your build speed.
bsub has a non-trivial startup time.
(unless LSF has gotten orders of magnitude faster).
distcc or icecream or other are going to yield much faster builds.

Are you building your list of sources via something like Glob("*.cc") ?

-Bill

On Tue, Oct 27, 2020 at 8:03 AM Jeremy Elson <jelson at gmail.com> wrote:

> Are you also migrating from a non-scons build system to scons? Or is it
> that scons drove distcc properly but now is spuriously rebuilding when
> you're using NFS files?
>
> In your example compiler invocation, you elided the build flags. Are they
> identical every time you run scons? scons rebuilds if the command line has
> changed at all, e.g. if you're including a build timestamp in the complier
> flags, or if the ordering of flags is not stable because you're storing
> them in a set.
>
> On Tue, Oct 27, 2020, 4:36 AM Stijn De Ruyck <Stijn.DeRuyck at onsemi.com>
> wrote:
>
>> Hello,
>>
>>
>>
>> We are in the process of migrating our development environment from
>> individual workstations to a compute grid. In other words from local files
>> & DistCC to NFS shares & LSF.
>>
>> We notice Scons 4.0.1 on Python 3.6.5/3.8.0 using the MD5 or
>> MD5-timestamp decider is always rebuilding all targets. Scons thinks all
>> dependencies have always changed even for what should be a no-op build.
>>
>>
>>
>> Source, target & external dependencies are on a single NFS share. Jobs
>> are being submitted by wrapping g++ in LSF’s bsub command in the Scons CXX
>> environment variable, similar to what we did with DistCC, which worked
>> fine, but with DistCC everything was stored on local disks…
>>
>>
>>
>> Example --debug=explain when rerunning on a previous successful build:
>>
>>
>>
>> scons: rebuilding `build/debug/base/Date.o'
>> because:
>>
>>            `base/Date.cc' is no longer a
>> dependency
>>
>>            `base/Date.cc' is a new
>> dependency
>>
>>
>>            `base/Date.icc'
>> changed
>>
>>
>>            `base/BaseTypes.h'
>> changed
>>
>>
>>  (…)
>>
>>            `base/String.icc'
>> changed
>>
>>
>>            `base/String.tcc'
>> changed
>>
>>
>>            `/site/custom/tools/lsf/prod/prod/linux2.6-glibc2.3-x86_64/bin/bsub'
>> changed
>>
>> bsub -I -n1 g++ -o build/debug/base/Date.o -c [...]
>> -I/devel/tme/deps/include -I. base/Date.cc
>>
>>
>>
>> Any advice? :/
>>
>>
>>
>> Regards,
>>
>>
>>
>> Stijn
>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20201027/6e5b6e5c/attachment-0001.html>