[Scons-users] timing issues and protecting from them

[Tom Tanner (BLOOMBERG/ LONDON)]

The only way really to guarantee a 100% build is to lock the filesystem against updates to any prerequeisite that is being used in a build rule, or when scanning a file for implicit dependencies. 

From: dl9obn at darc.de At: Dec  9 2015 14:57:30
To: scons-users at scons.org
Subject: Re: [Scons-users] timing issues and protecting from them

Hi Tom,

On 09.12.2015 14:15, Tom Tanner (BLOOMBERG/ LONDON) wrote:
> The problem is not in the generated file. It's in the source file.
>
> It's fine to change a source file outside scons (I hope). Just doing it while 
a build is in process is opening you up to race hazards.
>

but your approach of checking the timestamps of source files (or their MD5 
sums, or anything else) won't be able to guarantee a 
self-consistent and correct build either:

Let's say we have a target (library) Z, which depends on several sources A, B, 
C, D. So, after Z is built() you start to loop 
through the source dependencies in their lexicographical order. This checking 
will take time, and even if the single checks are 
relatively fast, it can still happen that you're currently checking C (with D 
still to go) and suddenly someone changes A from the 
outside. Then you get the same problem as before, an inconsistent state of 
timestamps vs hash.
Note how this problem is independent of which order you choose for processing 
the single children. You simply don't know the 
"correct" order beforehand...the one that you'd have to use for traversing the 
graph, such that the final modification happens 
before checking the corresponding source again.

So, from my perspective it looks as if guaranteeing that the build is 
self-consistent and correct, while allowing modifications from 
the outside, is impossible to do. What your approach definitely delivers is a 
slighter chance for a mishap like this...but I'm 
frankly more interested in solutions that give us the full 100% all the time 
(including parallel builds, which we haven't even 
touched yet ;)) instead of a 99.997% (I totally made that number up, but you 
get the idea).

Best regards,

Dirk

> From: wblevins001 at gmail.com At: Dec 9 2015 09:58:20
> To: Tom Tanner (BLOOMBERG/ LONDON) <mailto:ttanner2 at bloomberg.net>, 
scons-users at scons.org <mailto:scons-users at scons.org>
> Subject: Re: [Scons-users] timing issues and protecting from them
>
>     Read inline comments.
>
>     On Wed, Dec 9, 2015 at 8:11 AM, Tom Tanner (BLOOMBERG/ LONDON) 
<ttanner2 at bloomberg.net <mailto:ttanner2 at bloomberg.net>> wrote:
>
>         No. I'm implying it should happen at the time it does the build. Not 
during subsequent runs.
>
>         The situation we got into is:
>
>         timestamp in scons.dblite matches current timestamp of file
>         md5 in scons.dblite doesn't match current md5 of file
>
>         We have a somewhat paranoid decider function which checks for 
mismatchming md5s but that doesn't trigger if the timestamp
>         hasn't changed (as otherwise it would be very slow).
>
>         So the check I'm thinking of is something like in the .built function
>
>         if database timestamp of source dependency != actual timestamp of 
source dependency:
>         raise builderror (str(source) + ' changed during build')
>
>

_______________________________________________
Scons-users mailing list
Scons-users at scons.org
https://pairlist4.pair.net/mailman/listinfo/scons-users


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20151209/bd6f9f4f/attachment.html>