[Scons-users] How to schedule long actions earlier in the compile phase

[Don Baldwin]

Hi Bill,

Thanks for the quick response.  I believe our dependencies are setup correctly.  The problem happens when we have something like A depending on B, C, D and E, and D takes, let’s say, as long as all of the rest combined:

B, C and E each take 2 minutes
D takes 6 minutes
A is dependent on B, C, D and E

If we have -j set to 2, then we have the following scenarios:


  1.  Start D first, and while it’s running, we run B, C and E in parallel with it.  In which case the build takes 6 minutes before A can run.
  2.  Start B and C first, then run D and E in parallel.  In this case, E will end after 2 minutes leaving D to run by itself for 4 more minutes, and the whole thing takes 8 minutes before A can run.

I’ve tried to think of a way of using env.Depends to try to create some fake dependencies that would cause D to run first, but I can’t quite think of a way of doing this.  For example, I can’t make B, C and E dependent on D, because then SCons wouldn’t even start them until after D was finished.  I also tried to think of a way of adding a fake node that is dependent on D, and on which A would depend, but that would require SCons to look at the length of the dependency chains and run the longest ones first, and I don’t think it does that.

Anyway, thanks for looking into this.  Anything that you can think of would help.

Thanks,
Don


From: Scons-users <scons-users-bounces at scons.org> On Behalf Of Bill Deegan
Sent: Wednesday, September 15, 2021 10:19 AM
To: SCons users mailing list <scons-users at scons.org>
Subject: Re: [Scons-users] How to schedule long actions earlier in the compile phase


WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.
Don,

Not trivially.
Do these actions produce files used by other actions?

SCons doesn't really allow specifying order, Only dependencies.

Do these long actions depend on files generated by other builders?

-Bill

On Tue, Sep 14, 2021 at 8:26 PM Don Baldwin <donb at qti.qualcomm.com<mailto:donb at qti.qualcomm.com>> wrote:
Hi,

In our build, we have some particularly long actions that take about 10 minutes to complete.  If these get scheduled early during the compile phase of the build, there is no problem, but if they get scheduled later, then they become a long pole in the build, and keep running after everything else is complete.  This is causing some builds to take up to 5 minutes longer than others.

Is there a way to force SCons to run some actions before others as soon as they become available?  Maybe something like a priority queue?

Thanks,
Don Baldwin
_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20210915/4306637a/attachment-0001.htm>