[Scons-users] timing issues and protecting from them

[Tom Tanner (BLOOMBERG/ LONDON)]

The problem is not in the generated file. It's in the source file.

It's fine to change a source file outside scons (I hope). Just doing it while a build is in process is opening you up to race hazards.

From: wblevins001 at gmail.com At: Dec  9 2015 09:58:20
To: Tom Tanner (BLOOMBERG/ LONDON), scons-users at scons.org
Subject: Re: [Scons-users] timing issues and protecting from them

Read inline comments.

On Wed, Dec 9, 2015 at 8:11 AM, Tom Tanner (BLOOMBERG/ LONDON) <ttanner2 at bloomberg.net> wrote:

No. I'm implying it should happen at the time it does the build. Not during subsequent runs.

The situation we got into is:

  timestamp in scons.dblite matches current timestamp of file
  md5 in scons.dblite doesn't match current md5 of file

We have a somewhat paranoid decider function which checks for mismatchming md5s but that doesn't trigger if the timestamp hasn't changed (as otherwise it would be very slow).

So the check I'm thinking of is something like in the .built function

   if database timestamp of source dependency != actual timestamp of source dependency:
       raise builderror (str(source) + ' changed during build')

This contradicts your example case where the timestamp of the file and the timestamp in the scons.dblite match.

Seems that the only way to solve this issue would be to reevaluate nodes completely every time a node is used as a dependency. No one wants an extra O(N*M) md5sum computes.

The fundamental issue here is that files have been modified outside the SCons build it appears. This seems to be a common complaint, and my best advise is to not modify targets outside the SCons build context; don't mix and match build tools, etc. My best recommendation for trying to fix this situation is most cases is the patch that has already been presented.

Maybe someone else will have an idea.

V/R,
William
 


that shouldn't be an expensive check, and ensures what you have is self consistent.


From: scons-users at scons.org At: Dec  9 2015 00:00:43
To: scons-users at scons.org
Subject: Re: [Scons-users] timing issues and protecting from them

Are you implying this check should happen in subsequent runs?

Would you imagine it would look like https://bitbucket.org/scons/scons/pull-requests/240/added-support-for-detecting-corruption/diff ?

I haven't finished the patch, but you can at least see if it appears to do what you are asking.

On Tue, Dec 8, 2015 at 8:50 PM, Tom Tanner <trtanner at btinternet.com> wrote:

I've had an issue recently where someone updated a file while it was being used for a build (please, don't ask).

This results in an interesting sort of race hazard where the generated file information in the database says it's made from one version of a file, but it's actually made from another. If this happens it tends to result in errors, especially if that generated file ends up in your cache.

So it seemed to me that once you'd built a target it'd be a good idea to at least check all the prerequisites of the target hadn't changed their timestamp (as that would presumably be rather quicker than checking the md5 and just as effective) and generate an error.

But I'm not sure where the best place to  do that check would be or how to list all the prerequisites of a node. Has anyone come across this or know where I might want to start looking.

Cheers
_______________________________________________
Scons-users mailing list
Scons-users at scons.org
https://pairlist4.pair.net/mailman/listinfo/scons-users


         _______________________________________________
Scons-users mailing list
Scons-users at scons.org
https://pairlist4.pair.net/mailman/listinfo/scons-users
 


_______________________________________________
Scons-users mailing list
Scons-users at scons.org
https://pairlist4.pair.net/mailman/listinfo/scons-users


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20151209/74df76ad/attachment-0001.html>