[Scons-users] Not able to make full use of CPUs for processing speed

[Bill Deegan]

also, please consider joining the community on our discord server.
https://discord.gg/pejaFYrD9n

On Wed, Sep 15, 2021 at 4:08 PM Bill Deegan <bill at baddogconsulting.com>
wrote:

> Don,
>
> How big is your N?
> For reasonable N's SCons can keep up, but for N's in the 100s it' has some
> issues.
> The folks a MongoDB are using icecc and have built with N's > 250.. and
> SCons has issues.
> They've floated a PR to address a shortcoming in the scheduler which
> helped their build, but it's still marked a WIP so not merged.
> see: https://github.com/SCons/scons/pull/3386
>
> You could try making those changes locally and see if that helps.
>
> -Bill
>
>
> On Wed, Sep 15, 2021 at 3:45 PM Don Baldwin <donb at qti.qualcomm.com> wrote:
>
>> Hi,
>>
>>
>>
>> We have a build machine with, say, N CPUs, and we try increasing num_jobs
>> to a higher value, but we’re not seeing a decrease in build times.
>>
>>
>>
>> For example, say we have 10 CPUs.  We would expect that as num_jobs
>> approaches 10, we would continue to see better and better performance
>> during the compile phase.  However, what we’re seeing is that performance
>> improves until num_jobs hits about 5, at which point further increases to
>> num_jobs isn’t having an effect.
>>
>>
>>
>> My suspicion is that SCons is doing a lot of processing on it’s own
>> trying to figure out which jobs to queue next, e.g. scanning .c files for
>> new implicit dependencies, and that there’s a theoretical limit on how many
>> CPUs it’s able to keep up with.  Could something like that be going on?
>>
>>
>>
>> Thanks,
>>
>> Don
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20210915/54cf9060/attachment.htm>