[Scons-users] Bug causing corruption of .sconsign.dblite

[William Blevins]

Piotr,

Also, as a side question, do you have a set of test case values we can use
for this case?

V/R,
William

On Tue, Oct 14, 2014 at 7:23 PM, William Blevins <wblevins001 at gmail.com>
wrote:

> Piotr,
>
> I am sure you already did your homework, and we appreciate your input.
> Before we start looking at this closely, can you guarantee that the
> following files were not accidentally committed your GIT repository: object
> files, libraries, executables, the .sconsign.dblite file itself?  I have
> seen cases in the past were files like these get committed accidentally,
> and that might cause effects like those you described.
>
> V/R,
> William
>
> On Tue, Oct 14, 2014 at 10:14 AM, Piotr Bartosik <piotr.bartosik at gmail.com
> > wrote:
>
>> Hi,
>>
>> while using SCons (version 2.3.0) together with git I encountered some
>> problems with MD5 calculations. After rebasing to different revisions of my
>> repository I noticed strange cache misses (the proper .o files should be
>> already in the cache while they were actually rebuilt). This led me to the
>> conclusion that some MD5 values may be calculated incorrectly. In fact,
>> after using the script sconsign.py it became apparent that some MD5s in
>> .sconsign.dblite were incorrect (the sums calculated directly using hashlib
>> were different).
>>
>> To sum up, after this simple procedure:
>>
>> rm .sconsign.dblite
>> git checkout SOME_VERSION_1
>> scons
>> git checkout SOME_VERSION_2
>> scons
>>
>> sconsign.py reported incorrect MD5 sums.
>>
>> As it turned out, the problem is localized in Node/__init__.py. The code
>>
>> then.extend([None] * diff)
>>
>> in line 1050 causes desynchronization between two lists: the current
>> dependencies and info nodes for dependencies from the previous build
>> (containing, among others, previously calculated MD5 sums).
>>
>> For example, consider that in SOME_VERSION_1 a file has five dependencies:
>> A B C D E
>> and in SOME_VERSION_2 a new dependency appears:
>> A B C X D E
>>
>> After line 1050, the lists "then" and "children" look like below:
>> then:     A B C D E None
>> children: A B C X D E
>>
>> Code further below updates children information. As it appears, such
>> desynchronization does not cause any problems if we are checking MD5s each
>> time: they just will be recalculated for X, D and E. But if we also look at
>> the timestamps, the condition of timestamp remaining the same allows SCons
>> to copy the MD5 sum from the "then" array to the current child object.
>> Consequently, if D and E have the same modification date, E's checksum will
>> be incorrectly copied into D's info. And this is exactly the error that I
>> observe.
>>
>> My quick fix builds "then" list in a proper way, .i.e. without
>> desynchronization (please see the attachment):
>> then:     A B C None D E
>> children: A B C X    D E
>>
>> Please look at it and tell me what you think. I don't know the SCons'
>> sources very well so my fix may be a little naive but this is what I can
>> provide for a start.
>>
>> Thanks,
>> Piotr
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20141014/ab5fbc50/attachment.html>