[Scons-users] Prioritization of builds

[William Blevins]

On Sat, Jan 16, 2016 at 1:51 AM, Evan Driscoll <driscoll at cs.wisc.edu> wrote:

> I have a similar situation to Brian Cody, where if the wrong source files
> get updated, I can wind up with a 30-minute build that does *almost
> everything* in the first 15 minutes but that spends another 15 minutes
> serially running a ton of targets. I've *also* wished that I could say
> "prioritize these targets" so it would build the must-be-done-serially
> targets ASAP and then build everything else around it.
>

Is the SCons DAG parser DFS or BFS or other? Seems that it is DFS, and may
see better parallelism with BFS?


>
> The 'scons <those targets>; scons' script idea wouldn't really help in my
> case; the benefit would be from the fact that much of the first 15 minutes
> *could* be done in parallel with everything from the second 15 minutes.
>
> The "priority" thing feels pretty natural to me and not anti-SCons...
> you're still not anywhere close to saying "you build this by taking these
> steps." You're still specifying the DAG, you're still letting dependency
> resolution do it's thing, you're just saying "build these targets first." I
> guess something that automatically recorded the time and then tried to pick
> the best scheduling would be a bit clearer, but I don't think it's
> necessary to keep the flavor.
>
>
> On a somewhat but not entirely unrelated note, the other thing that would
> be sometimes helpful to me is a *memory* aware version of -j. On many of
> our machines, we are memory constrained rather than CPU constrained. So the
> question then comes up with what do you pass as -j? If you pass a high
> number, you start thrashing when it hits the larger files; if you pass a
> small number, there's a lot of lost parallelism during much of the build.
> ('scons <not-memory-constrained targets> -j8; scons -j4' *does* work very
> well for this problem.)
>
> I've actually thought about and at one point started writing a wrapper for
> gcc that just passes through to real gcc but doesn't invoke the compiler
> until the memory use is below a threshold, but never got around to getting
> everything to work with it. (I don't remember what the problem was.) Also
> thought about making the wrapper silently kill and re-invoke the compiler
> if the memory use started getting too high.
>
> This is another place where, if SCons recorded memory use of the build
> steps, it could use that information to predict a better build order.
> Though I recognize that would be a ton of work.
>
>
> (I present these things largely as brainstorming; if I can plant the idea
> in the back of your mind or advocate for the priority thing enough that it
> makes it in, that would be helpful I think.)
>
> Evan
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20160116/ab99c9fb/attachment.html>