[Scons-users] Bug causing corruption of .sconsign.dblite

[Piotr Bartosik]

William,

thank you for your answer. I'm stating to prepare a test data set. I'm
going to send you a compressed directory versioned with git (with .git dir
inside) and an exact list of steps to reproduce the problem. I hope that
this format is convenient to you, if not please let me know.

Best regards,
Piotr


2014-10-15 1:25 GMT+02:00 William Blevins <wblevins001 at gmail.com>:

> Piotr,
>
> Also, as a side question, do you have a set of test case values we can use
> for this case?
>
> V/R,
> William
>
> On Tue, Oct 14, 2014 at 7:23 PM, William Blevins <wblevins001 at gmail.com>
> wrote:
>
>> Piotr,
>>
>> I am sure you already did your homework, and we appreciate your input.
>> Before we start looking at this closely, can you guarantee that the
>> following files were not accidentally committed your GIT repository: object
>> files, libraries, executables, the .sconsign.dblite file itself?  I have
>> seen cases in the past were files like these get committed accidentally,
>> and that might cause effects like those you described.
>>
>> V/R,
>> William
>>
>> On Tue, Oct 14, 2014 at 10:14 AM, Piotr Bartosik <
>> piotr.bartosik at gmail.com> wrote:
>>
>>> Hi,
>>>
>>> while using SCons (version 2.3.0) together with git I encountered some
>>> problems with MD5 calculations. After rebasing to different revisions of my
>>> repository I noticed strange cache misses (the proper .o files should be
>>> already in the cache while they were actually rebuilt). This led me to the
>>> conclusion that some MD5 values may be calculated incorrectly. In fact,
>>> after using the script sconsign.py it became apparent that some MD5s in
>>> .sconsign.dblite were incorrect (the sums calculated directly using hashlib
>>> were different).
>>>
>>> To sum up, after this simple procedure:
>>>
>>> rm .sconsign.dblite
>>> git checkout SOME_VERSION_1
>>> scons
>>> git checkout SOME_VERSION_2
>>> scons
>>>
>>> sconsign.py reported incorrect MD5 sums.
>>>
>>> As it turned out, the problem is localized in Node/__init__.py. The code
>>>
>>> then.extend([None] * diff)
>>>
>>> in line 1050 causes desynchronization between two lists: the current
>>> dependencies and info nodes for dependencies from the previous build
>>> (containing, among others, previously calculated MD5 sums).
>>>
>>> For example, consider that in SOME_VERSION_1 a file has five
>>> dependencies:
>>> A B C D E
>>> and in SOME_VERSION_2 a new dependency appears:
>>> A B C X D E
>>>
>>> After line 1050, the lists "then" and "children" look like below:
>>> then:     A B C D E None
>>> children: A B C X D E
>>>
>>> Code further below updates children information. As it appears, such
>>> desynchronization does not cause any problems if we are checking MD5s each
>>> time: they just will be recalculated for X, D and E. But if we also look at
>>> the timestamps, the condition of timestamp remaining the same allows SCons
>>> to copy the MD5 sum from the "then" array to the current child object.
>>> Consequently, if D and E have the same modification date, E's checksum will
>>> be incorrectly copied into D's info. And this is exactly the error that I
>>> observe.
>>>
>>> My quick fix builds "then" list in a proper way, .i.e. without
>>> desynchronization (please see the attachment):
>>> then:     A B C None D E
>>> children: A B C X    D E
>>>
>>> Please look at it and tell me what you think. I don't know the SCons'
>>> sources very well so my fix may be a little naive but this is what I can
>>> provide for a start.
>>>
>>> Thanks,
>>> Piotr
>>>
>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>>>
>>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20141015/bbb5e307/attachment-0001.html>