[Scons-users] Bug causing corruption of .sconsign.dblite

[William Blevins]

Piotr,

That should be fine.  Try to keep the dataset as simple and lightweight as
possible, since a test case needs to be created from it.  If you only need
a single c-file with a few header includes to reproduce, then that should
be plenty.

If you feel comfortable request a patch addition, you can do so here:
https://bitbucket.org/scons/scons.

V/R,
William

On Wed, Oct 15, 2014 at 5:13 AM, Piotr Bartosik <piotr.bartosik at gmail.com>
wrote:

> William,
>
> thank you for your answer. I'm stating to prepare a test data set. I'm
> going to send you a compressed directory versioned with git (with .git dir
> inside) and an exact list of steps to reproduce the problem. I hope that
> this format is convenient to you, if not please let me know.
>
> Best regards,
> Piotr
>
>
> 2014-10-15 1:25 GMT+02:00 William Blevins <wblevins001 at gmail.com>:
>
>> Piotr,
>>
>> Also, as a side question, do you have a set of test case values we can
>> use for this case?
>>
>> V/R,
>> William
>>
>> On Tue, Oct 14, 2014 at 7:23 PM, William Blevins <wblevins001 at gmail.com>
>> wrote:
>>
>>> Piotr,
>>>
>>> I am sure you already did your homework, and we appreciate your input.
>>> Before we start looking at this closely, can you guarantee that the
>>> following files were not accidentally committed your GIT repository: object
>>> files, libraries, executables, the .sconsign.dblite file itself?  I
>>> have seen cases in the past were files like these get
>>> committed accidentally, and that might cause effects like those you
>>> described.
>>>
>>> V/R,
>>> William
>>>
>>> On Tue, Oct 14, 2014 at 10:14 AM, Piotr Bartosik <
>>> piotr.bartosik at gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> while using SCons (version 2.3.0) together with git I encountered some
>>>> problems with MD5 calculations. After rebasing to different revisions of my
>>>> repository I noticed strange cache misses (the proper .o files should be
>>>> already in the cache while they were actually rebuilt). This led me to the
>>>> conclusion that some MD5 values may be calculated incorrectly. In fact,
>>>> after using the script sconsign.py it became apparent that some MD5s in
>>>> .sconsign.dblite were incorrect (the sums calculated directly using hashlib
>>>> were different).
>>>>
>>>> To sum up, after this simple procedure:
>>>>
>>>> rm .sconsign.dblite
>>>> git checkout SOME_VERSION_1
>>>> scons
>>>> git checkout SOME_VERSION_2
>>>> scons
>>>>
>>>> sconsign.py reported incorrect MD5 sums.
>>>>
>>>> As it turned out, the problem is localized in Node/__init__.py. The code
>>>>
>>>> then.extend([None] * diff)
>>>>
>>>> in line 1050 causes desynchronization between two lists: the current
>>>> dependencies and info nodes for dependencies from the previous build
>>>> (containing, among others, previously calculated MD5 sums).
>>>>
>>>> For example, consider that in SOME_VERSION_1 a file has five
>>>> dependencies:
>>>> A B C D E
>>>> and in SOME_VERSION_2 a new dependency appears:
>>>> A B C X D E
>>>>
>>>> After line 1050, the lists "then" and "children" look like below:
>>>> then:     A B C D E None
>>>> children: A B C X D E
>>>>
>>>> Code further below updates children information. As it appears, such
>>>> desynchronization does not cause any problems if we are checking MD5s each
>>>> time: they just will be recalculated for X, D and E. But if we also look at
>>>> the timestamps, the condition of timestamp remaining the same allows SCons
>>>> to copy the MD5 sum from the "then" array to the current child object.
>>>> Consequently, if D and E have the same modification date, E's checksum will
>>>> be incorrectly copied into D's info. And this is exactly the error that I
>>>> observe.
>>>>
>>>> My quick fix builds "then" list in a proper way, .i.e. without
>>>> desynchronization (please see the attachment):
>>>> then:     A B C None D E
>>>> children: A B C X    D E
>>>>
>>>> Please look at it and tell me what you think. I don't know the SCons'
>>>> sources very well so my fix may be a little naive but this is what I can
>>>> provide for a start.
>>>>
>>>> Thanks,
>>>> Piotr
>>>>
>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>>>
>>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20141015/10ecfe50/attachment.html>