[Scons-users] Making some tools run only one at a time

[Plunket, Tom]

I wanted to follow up to say that I've tried this and it almost works but I haven't spent the time yet to really dig into why it is broken.

The below does in fact make subsequent jobs for the same tool queue up rather than just charge forward but then it seems to be modifying the filename of the thing that is dependent on the dummy asset.

I.e. I'm managing all of my own filenames so I'm doing:

env.MyTool(output, input)
env.SideEffect('#/dummy-lock.txt', output)

But while MyTool runs correctly and successfully, I get an error that states SCons can't find the output file on a different path than the one it's been written to.

I'll look into this deeper though before I engage anyone further though. I feel like I'm pretty far off of the reservation with the stuff I'm doing so more knowledge of the SCons internals will help me talk intelligently.


-Tom


From: scons-users-bounces at scons.org [mailto:scons-users-bounces at scons.org] On Behalf Of Bill Deegan
Sent: Friday, March 07, 2014 3:04 PM
To: SCons users mailing list
Subject: Re: [Scons-users] Making some tools run only one at a time

Tom,
Add a SideEffect() to your builders such that all of them would have the same file as a sideeffect.
This should prevent more than one from running at a time.
The file doesn't actually have to get created.
-Bill

On Fri, Mar 7, 2014 at 2:48 PM, Plunket, Tom <tom.plunket at aristocrat-inc.com> wrote:
Hey gang,

I've searched the archives but haven't come up with anything; apologies
if this is obvious somewhere but I haven't seen it.

I'm building a data processing pipeline backed by SCons. I've defined a
few tools to do the things that need to be done but I would like to flag
a couple of them as "only run one of these at a time."

SCons is the framework that I'm relying on as I split a
mega-do-everything tool into smaller tools that do different aspects of
the build job. One stage of the pipeline opens up a heavyweight art
package and drives an export of the data into a file format that I can
process elsewhere. Another stage of the pipeline will take a list of
bitmaps and send them to a video compressor.

It doesn't really matter what the tools are but the problem I'm looking
to address is that when I multithread my build, it starts with the art
export and so it fires up eight (on my machine) instances of this
heavyweight art package when running them serially (or better yet,
collecting all of the files that need to be exported and then just
process them all serially in one instance of the tool) would actually be
prefereable. And then on the video compressor tool, that tool already
takes advantage of multiple cores so I'd like to only execute those one
at a time as well. Certianly I could put a wait into the generator but
that would presumably prevent other things from running.

Is there a way to do this that I've missed? Even maybe something like
returning from a generator, "not yet, homeboy, wait your turn"?

Thanks.





IMPORTANT CONFIDENTIALITY NOTICE:

This E-mail (including any documents referred to in, or attached, to this E-mail) may contain information that is personal, confidential or the subject of copyright or other proprietary rights in favor of Aristocrat, its affiliates or third parties. This E-mail is intended only for the named addressee. Any privacy, confidence, copyright or other proprietary rights in favor of Aristocrat, its affiliates or third parties, is not lost because this E-mail was sent to you by mistake.

If you received this E-mail by mistake you should: (i) not copy, disclose, distribute or otherwise use it, or its contents, without the consent of Aristocrat or the owner of the relevant rights; (ii) let us know of the mistake by reply E-mail or by telephone (US 1-877-274-9661, or  AU +61 2 9013 6000); and (iii) delete it from your system and destroy all copies.

Any personal information contained in this E-mail must be handled in accordance with applicable privacy laws.

Electronic and internet communications can be interfered with or affected by viruses and other defects. As a result, such communications may not be successfully received or, if received, may cause interference with the integrity of receiving, processing or related systems (including hardware, software and data or information on, or using, that hardware or software). Aristocrat gives no assurances in relation to these matters.

If you have any doubts about the veracity or integrity of any electronic communication we appear to have sent you, please call (US 1-877-274-9661, or  AU +61 2 9013 6000) for clarification.
_______________________________________________
Scons-users mailing list
Scons-users at scons.org
http://four.pairlist.net/mailman/listinfo/scons-users



IMPORTANT CONFIDENTIALITY NOTICE:

This E-mail (including any documents referred to in, or attached, to this E-mail) may contain information that is personal, confidential or the subject of copyright or other proprietary rights in favor of Aristocrat, its affiliates or third parties. This E-mail is intended only for the named addressee. Any privacy, confidence, copyright or other proprietary rights in favor of Aristocrat, its affiliates or third parties, is not lost because this E-mail was sent to you by mistake.

If you received this E-mail by mistake you should: (i) not copy, disclose, distribute or otherwise use it, or its contents, without the consent of Aristocrat or the owner of the relevant rights; (ii) let us know of the mistake by reply E-mail or by telephone (US 1-877-274-9661, or  AU +61 2 9013 6000); and (iii) delete it from your system and destroy all copies.

Any personal information contained in this E-mail must be handled in accordance with applicable privacy laws.

Electronic and internet communications can be interfered with or affected by viruses and other defects. As a result, such communications may not be successfully received or, if received, may cause interference with the integrity of receiving, processing or related systems (including hardware, software and data or information on, or using, that hardware or software). Aristocrat gives no assurances in relation to these matters.

If you have any doubts about the veracity or integrity of any electronic communication we appear to have sent you, please call (US 1-877-274-9661, or  AU +61 2 9013 6000) for clarification.