[Scons-users] MD5-timestamp decider overrides csigs from SConsign incorrectly

[Sean Houghton]

We’ve been having a difficult-to-track-down issue which we believe boils
down to the behavior of the MD5-timestamp decider when the list of
dependencies has changed between current runs and previous runs.



The main Node.changed function calls the decider function for every input
to a node in all cases (called out by a comment “Note that we now *always*
check every dependency”). The inputs are determined by zip()ing together
two lists: a list of current children and a list of previous ninfos. This
happens even if the length of the list or the order of dependencies in the
list has changed:



    diff = len(children) - len(then)

    if diff:

        # The old and new dependency lists are different lengths.

        # This always indicates that the Node must be rebuilt.

        # We also extend the old dependency list with enough None

        # entries to equal the new dependency list, for the benefit

        # of the loop below that updates node information.

        then.extend([None] * diff)

        if t: Trace(': old %s new %s' % (len(then), len(children)))

        result = True



    for child, prev_ni in zip(children, then):

        if child.changed_since_last_build(self, prev_ni):

            if t: Trace(': %s changed' % child)

            result = True



The implementation of `changed_since_last_build` on FS.File nodes for
MD5-timestamp overwrites the csig of a node whenever the timestamp matches:



    def changed_timestamp_then_content(self, target, prev_ni):

        if not self.changed_timestamp_match(target, prev_ni):

            try:

                self.get_ninfo().csig = prev_ni.csig

            except AttributeError:

                pass

            return False

        return self.changed_content(target, prev_ni)



If `prev_ni` does not correspond to the current node, the timestamps are
checked anyway. If the timestamps happen to match (one second granularity
is easily coarse enough for this to be possible), then the node’s current
csig is blindly overwritten by the csig pulled from `prev_ni`--even if it
belongs to a completely different node!



In our setup, which uses CacheDir, this led to bad csigs and a bad
“cachedir_bsig”, which led to retrieving the wrong file from the cache
directory when the set of dependencies for a node changed.



Can someone confirm this a SCons bug? We're switching back to the basic MD5
decider for now. We'll also spend some time and attempt to fix it as soon
as possible. (Note that we’re on 2.3.6, but I checked the BitBucket repo
and the current version appears to have the same issue.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20160114/3471074d/attachment.html>