[Scons-users] modifying/extending SCons to run jobs on Torque cluster

[Bill Deegan]

Thomas,

Can't torque submit jobs and wait or their completion?
If so the master SCons's process could maintain enough information and the
need to merge the sconsigns would be removed..

-Bill

On Sat, Oct 4, 2014 at 1:43 PM, Thomas Lippincott <tom.lippincott at gmail.com>
wrote:

> This would be OK for my purposes, if "global" means a given invocation
> of the primary scons script. When scons is invoked on the submit node,
> it might decide to build the tree by submitting three jobs, which run
> "scons job1-target1 job1-target2...", "scons job2-target1
> job2-target2...", etc, on worker nodes.  The submit-node's scons
> instance copies ".sconsign.db" to ".sconsign.JOB-1", ".sconsign.JOB-2",
> etc, then submits the jobs to the scheduler and starts waiting for them
> to complete.  Meanwhile, worker nodes execute the jobs, but when running
> on a worker node the scons script first calls
> SConsignFile(".sconsign.JOB-ID").  Back at the submit-node's scons
> instance, once the worker nodes have finished, scons needs to read all
> the ".sconsign.JOB-ID" files and merge them back into ".sconsign.db". It
> sounds like this final merging functionality is the main work (in theory).
> -Tom
>
>
> On 10/04/2014 02:13 PM, Bill Deegan wrote:
> > The SConsignFile() is global right?
> > I don't think you can do that per environment or per builder.
> >
> > -Bill
> >
> > On Sat, Oct 4, 2014 at 1:46 AM, Dirk Bächle <tshortik at gmx.de> wrote:
> >
> >>  Tom,
> >>
> >> On 04.10.2014 06:42, Bill Deegan wrote:
> >>
> >>  Is there functionality for directing the scons command to
> >> use a different .sconsign.db file, and an API for
> >> reading/merging/writing entries from different files?
> >>
> >>  No. (Unless of course you write it.. :)
> >>
> >>
> >> there is the SConsignFile() method for redirecting the writing/reading
> of
> >> entries to a different filename. The rest of the current API can be
> found
> >> in "engine/SCons/SConsign.py"...
> >>
> >> Dirk
> >>
> >>
> >> _______________________________________________
> >> Scons-users mailing list
> >> Scons-users at scons.org
> >> https://pairlist4.pair.net/mailman/listinfo/scons-users
> >>
> >>
> >
> >
> >
> > _______________________________________________
> > Scons-users mailing list
> > Scons-users at scons.org
> > https://pairlist4.pair.net/mailman/listinfo/scons-users
> >
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20141004/191e5a43/attachment-0001.html>