[Scons-users] parallel invocation of SCons

[Herzog, Tobias (CQSP)]

Hi Bill,



the 99% solution: So it is sufficient, to lock only during execution of the SConstruct/SConscript file? My picture was, that SCons is writing/reading to its database after all the SConstruct/SConsctript are read, or am I wrong?



Tobias

Von: Scons-users <scons-users-bounces at scons.org> Im Auftrag von Bill Deegan
Gesendet: Donnerstag, 14. Februar 2019 17:56
An: SCons users mailing list <scons-users at scons.org>
Betreff: Re: [Scons-users] parallel invocation of SCons

Tobias,

I'd say sorting out your CI to avoid doing this would be the best solution.
Which CI system are you using?
Anything else can get pretty complicated and not necessary be a 100% solution.
(Think race.. they start at (close enough to) the same time).

If you can live with a 99% solution, the just do something like this
SConstruct:
# before you do anything
check if file exists, if so sleep and wait until it's gone
Then create the "lockfile"
< your normal SConstruct logic>
# at the bottom of the file
remove your lockfile

This should be simpler than an interprocess lock and also work if your build workers share a file system but an run on multiple machines.

That said, I'd really suggest you handle this in your CI.

-Bill

On Thu, Feb 14, 2019 at 1:19 AM Herzog, Tobias (CQSP) <tobias.herzog at carmeq.com<mailto:tobias.herzog at carmeq.com>> wrote:
Hi SCons users,

due to a parallelization in a CI build pipeline, I have the use case, that SCons (possibly) is invoked concurrently in the same tree (i.e. using database) but with different build targets. I already noticed, that this can lead to a crashing SCons and/or corrupted database.
For me it would be sufficient, if SCons just waits until the other SCons process has finished. My Idea was, to acquire any kind of inter process lock right in the SConstruct, that is active until the process terminates. So my assumption here is, that database access takes place after the reading of SConstruct/SConscript is done.
Is this correct? Will this solution be safe? Has anyone solved this problem in another/better way?

Thanks an best regards,
Tobias
_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20190218/0d045d4b/attachment.html>