[Scons-users] Fwd:Re: timing issues and protecting from them

[Tom Tanner]

On 9/12/15 17:00, Dirk Bächle wrote:
> Tom,
>
> On 09.12.2015 17:22, Tom Tanner (BLOOMBERG/ LONDON) wrote:
>>
>> [...]
>>
>>     I don't think this is a problem.
>>
>
> okay so let's go with your example then...
>
>> [...]
>>
>>     There is no need to have a correct order. If *any* timestamp has 
>> changed between deciding the target needed building and
>>     finishing the check of all the timestamps, then you have a 
>> potential problem. OK, you might get a false positive if a file
>>     timestamp changed right at the end. But you won't get a false 
>> negative.
>>
>
> My point here is that you're not always able to detect that the 
> timestamp actually changed, based on the fact that you have to choose 
> a processing order. And this order then might interfere with what's 
> actually happening...
>
>>     Also, the issue is NOT that you have a library. The point is that
>>
>>     1) You have a library that depends on a.o, b.o, c.o
>>     2) a.o depends on a.cc, a.hh, b.hh; etc etc
>>
>>     So when you build a.o, before you decide the build was succesful, 
>> you check the times on a.cc, a.hh, b.hh
>>
>
> So, you have checked the times of "a.cc" and "a.hh" already, and are 
> about to process "b.hh". Now, suddenly someone from the outside 
> changes "a.cc", and you won't detect this timestamp change. Your build 
> is inconsistent, depending on how you define "consistency".
> You might say: That's a case for the next build, because the file was 
> changed *after* checking. But then what's your "horizon" (can't think 
> of a better word right now) for checking. Until when is it allowed to 
> change a file, and when do you say: "That change has to be handled by 
> the next build"?
>
This is why I say - have any of the sources changed *while* I was 
building the target. This you check *after* the target has been built. 
So the database currently says

a.hh - md5 MAH, time TAH
a.cc - md5 MAC, time TAC
b.hh - md5 MBH, time TBH

then you execute the build actions to generate a.o

then you check if any of a.hh, a.cc, b.hh have changed (on disk) from 
the values you currently have stored.

And if they have, then you no longer have the right information to 
determine what the pre-requisites of a.o are, because you don't know 
when whatever changed did change. The prerequisite could have changed 
before the build action used the contents or it could have changed 
after. And if you pick the wrong one, you'll get incorrect information 
stored. And you shouldn't in any case store the changed time for a.hh 
because that could break the database for other targets that you've 
already built that also depend on a.hh. So the only thing you can do is 
generate an error.

If however, a.hh changes *after* this point, you don't need to worry 
because you know the next build will identify the fact that a.hh is 
different when it evaluates whether or not it needs to build a.o, 
because you will have stored the correct information about the state of 
the prerequisites anyway.

So basically your procedure for building seems to need to be something like:

     I have target T to build from S1, S2, S3

    Each Si has a timestamp and md5sum. There is the one from the 
previous build and the one we checked when re-evaluating the tree. Both 
these are immutable once calculated.

    Do T's build action

    Check each Si to see if it has changed. If so generate an error.

    Store T's pre-requisites. These will refer to the immutable timestamps.

There is no horizon you have to worry about. Either the file is 
different between before the build action and after the build action, in 
which case you have a problem, or it isn't, in which case you know the 
build was done with the versions of the files you think it was built 
with and all is OK.

It doesn't matter if you then do 0, 5, 10, 500 or even 5000 other build 
actions and change a.hh at some point during that (and presuming they 
don't depend on a.hh either!). Because T says 'I depend on a.hh with xxx 
timestamp/md5' and next time you do a build, scons will see that a.hh 
has changed.

But this sequence will not work, which is what you seem to be suggesting 
is happening:

     get time/md5 of a.hh, b.hh, a.cc
     do build action
     check a.hh
     check b.hh
     check a.cc
     <---- a.hh changes externally
     store *current on disk* information of a.hh, b.hh, a.cc (with the 
new timestamp of a.hh)

  If that is what happens, that's going to go horribly wrong.

Basically I suppose you're allowed to change a file right up until scons 
has looked at it and decided it has changed, and then you shouldn't 
change it until scons has built everything that directly depends on it.