[Scons-users] timing issues and protecting from them

[Dirk Bächle]

Hi Tom,

On 09.12.2015 14:15, Tom Tanner (BLOOMBERG/ LONDON) wrote:
> The problem is not in the generated file. It's in the source file.
>
> It's fine to change a source file outside scons (I hope). Just doing it while a build is in process is opening you up to race hazards.
>

but your approach of checking the timestamps of source files (or their MD5 sums, or anything else) won't be able to guarantee a 
self-consistent and correct build either:

Let's say we have a target (library) Z, which depends on several sources A, B, C, D. So, after Z is built() you start to loop 
through the source dependencies in their lexicographical order. This checking will take time, and even if the single checks are 
relatively fast, it can still happen that you're currently checking C (with D still to go) and suddenly someone changes A from the 
outside. Then you get the same problem as before, an inconsistent state of timestamps vs hash.
Note how this problem is independent of which order you choose for processing the single children. You simply don't know the 
"correct" order beforehand...the one that you'd have to use for traversing the graph, such that the final modification happens 
before checking the corresponding source again.

So, from my perspective it looks as if guaranteeing that the build is self-consistent and correct, while allowing modifications from 
the outside, is impossible to do. What your approach definitely delivers is a slighter chance for a mishap like this...but I'm 
frankly more interested in solutions that give us the full 100% all the time (including parallel builds, which we haven't even 
touched yet ;)) instead of a 99.997% (I totally made that number up, but you get the idea).

Best regards,

Dirk

> From: wblevins001 at gmail.com At: Dec 9 2015 09:58:20
> To: Tom Tanner (BLOOMBERG/ LONDON) <mailto:ttanner2 at bloomberg.net>, scons-users at scons.org <mailto:scons-users at scons.org>
> Subject: Re: [Scons-users] timing issues and protecting from them
>
>     Read inline comments.
>
>     On Wed, Dec 9, 2015 at 8:11 AM, Tom Tanner (BLOOMBERG/ LONDON) <ttanner2 at bloomberg.net <mailto:ttanner2 at bloomberg.net>> wrote:
>
>         No. I'm implying it should happen at the time it does the build. Not during subsequent runs.
>
>         The situation we got into is:
>
>         timestamp in scons.dblite matches current timestamp of file
>         md5 in scons.dblite doesn't match current md5 of file
>
>         We have a somewhat paranoid decider function which checks for mismatchming md5s but that doesn't trigger if the timestamp
>         hasn't changed (as otherwise it would be very slow).
>
>         So the check I'm thinking of is something like in the .built function
>
>         if database timestamp of source dependency != actual timestamp of source dependency:
>         raise builderror (str(source) + ' changed during build')
>
>