[Scons-users] Forcing projects to build in order

Fri Aug 7 09:59:50 EDT 2015

Tobbe,

There are a number of different issues you bring up below addressed in line.

-Bill

On Fri, Aug 7, 2015 at 3:54 AM, Sven Lars Tobias Stål <smp428 at alumni.ku.dk>
wrote:

> Dear all,
>
> I’d like to second that wish. The option to pin processes to run first or
> last would be very useful also in other applications, especially for us who
> are not experts in compilation and construction tools.
>

Generally speaking forcing order is not what SCons is meant to do. That
said in some instances it's not too hard to do, or to resolve whatever
issue is making you want to force order rather than hacking around an issue.

>
> I’m using Scons only to build reproducible processing flows for
> geophysical data. SCons is really great for the purpose of many reasons!
> However, processing times are often long and it would be fantastic to be
> able to e.g. run a python code to notify when it’s done, or the run a
> conversion program for all results, without having to name them all.
>

As for running conversion code on all results, might I suggest that you
output all these into a specific output directory and make sure you either
list the targets in the command or have an emitter for the builder, then
you can use env.Glob() on that output dir to gather the sources to run
through your conversion program.

> A process pinned to the beginning of the SConstruct flow would be useful
> to inform the user of the process and, especially, check some parameters
> instead of forcing halt in the middle of the process, when I’m not in
> office.
>

If you wish to have some logic you can run on command rather than having to
run the whole build you can use:
list_of_nodes=[]
list_of_nodes.extend(env.Command(Command #1...))
list_of_nodes.extend(env.Command(Command #2...))
..
env.Alias('some-target-name', list_of_nodes)

If this is just some straight python logic and not commands, as Gary said
you can just run them in the SConstruct/SConscript and not use builders to
add them to the DAG..

> Some Boolean operations to name files and set parameters would also be
> very useful to run before the construction starts and it would be useful to
> force a file to be processed first, so that I can check the result without
> running all processes.
>

See Alias above..

>
> For now, I’m managing by constructing empty files that have ‘artificial'
> dependencies, but it causes confusion for other users - and for me and adds
> a lot of lines. I also tried to run SConst as a subprocess to a  script,
> but that really feels like complicating things.
>

Yup. You can run into issues with the sconsign file with this..

>
> I do understand that such function is somehow violating the principle of
> dependency order, but I’d guess that it could be easy to implement and, for
> those who are better at using construction tools, easy to just to leave
> out.
>
> All the best
> Tobbe
>
>
>
> On 07 Aug 2015, at 11:22 , Ora <orshalev at gmail.com> wrote:
>
> Hi,
>
> During a few of the build steps the script calls various tools which MUST
> be run in a specific directory.  That is not something I can change at this
> point.
>
> Each such step is defined as its own builder which changes the directory
> and runs the required tool.
>
> However, when running with the '-j' option, Scons tries to run several
> such steps in parallel, and therefore, tries to execute commands in the
> wrong directory.
>
> I cannot avoid parallelism altogether as it it would make my script run
> for so long, it would be useless.
> I cannot avoid changing directories.
>
> I wish SCons would handle changing directories properly on its own, but I
> understand that's not going to happen.
>
> I am therefore looking for a solution to this problem by partially
> limiting parallelism so that the builder steps / projects which change
> directory would not be called in parallel with other builders.
>
>
> Ora
>
> On Thu, Aug 6, 2015 at 7:11 PM, William Blevins <wblevins001 at gmail.com>
> wrote:
>
>>
>> On Thu, Aug 6, 2015 at 11:09 AM, Bill Deegan <bill at baddogconsulting.com>
>> wrote:
>>
>>>
>>>
>>> On Thu, Aug 6, 2015 at 8:04 AM, William Blevins <wblevins001 at gmail.com>
>>> wrote:
>>>
>>>> I cannot think of many (if any) good reasons to perform CWD.
>>>>
>>>> Can you modify your build to work without?
>>>> On Thu, Aug 6, 2015 at 10:26 AM, Bill Deegan <bill at baddogconsulting.com
>>>> > wrote:
>>>>
>>>>> Ora,
>>>>>
>>>>> Why do you want to force build order and/or limit parallelism?
>>>>>
>>>>>
>>>> He is performing "change working directory" commands which cause issues
>>>> with parallel builds.
>>>>
>>>
>>> When I run into those I typically do env.Command(target,source,"cd xyz;
>>> do actual command")  or wrap said command in a script and run that from the
>>> command.
>>>
>>
>> That makes sense.  Keeps the CWD commands outside the SCons context :)
>>
>>
>>>
>>>>
>>>>> -Bill
>>>>>
>>>>> On Thu, Aug 6, 2015 at 3:58 AM, Ora <orshalev at gmail.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> My script builds several different projects which are not necessarily
>>>>>> related.   How can I force those projects to build in a certain order?
>>>>>>
>>>>>> That is, I have three projects: a, b and c.
>>>>>> I want 'b' to start building only after 'a' finishes, and for 'c' to
>>>>>> start building only after 'b' finishes.
>>>>>>
>>>>>> This is necessary in order to avoid problems with changing
>>>>>> directories during parallel builds.
>>>>>>
>>>>>> I tried using 'Requires', with every target requiring the target
>>>>>> before it.
>>>>>> Requires (b, a)
>>>>>> Requires (c, b)
>>>>>>
>>>>>> But that doesn't solve the problem because, if I understood
>>>>>> correctly, it only tells Scons that it should finish building 'a' before it
>>>>>> finishes building 'b', and doesn't prevent it from compiling both projects
>>>>>> at the same time.
>>>>>>
>>>>>> Any suggestions on how to deal with this issue would be appreciated.
>>>>>>
>>>>>> Thanks in advance,
>>>>>> Ora
>>>>>>
>>>>>> _______________________________________________
>>>>>>
>>>>>
>>>
>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20150807/9aa55fde/attachment-0001.html>