[Scons-users] Making SCons deal more robustly with incremental linking

Wed Jun 10 02:24:48 EDT 2015

Please try the following patch.  If this works for you, I will make a
regression test and submit a pull request.

--- a/src/engine/SCons/Node/__init__.py    Sun May 17 23:38:23 2015 -0400
+++ b/src/engine/SCons/Node/__init__.py    Wed Jun 10 02:20:32 2015 -0400
@@ -1083,9 +1083,12 @@
         if node is None:
             node = self

-        result = False
+        bi = node.get_stored_info().binfo
+        ni = node.get_stored_info().ninfo

-        bi = node.get_stored_info().binfo
+        result = node.get_csig() != ni.csig
+        if result and t: Trace('csig modified.')
+
         then = bi.bsourcesigs + bi.bdependsigs + bi.bimplicitsigs
         children = self.children()


V/R,
William

On Wed, Jun 10, 2015 at 1:52 AM, William Blevins <wblevins001 at gmail.com>
wrote:

> Looks like the issue exists in class Node method changed().
>
> The function does not evaluate whether its signature has changed. At a
> glance, I'm not sure how to perform the check, but it should be simple
> since the md5sum or other metadata should already be stored somewhere...
>
> V/R,
> William
>
> On Tue, Jun 9, 2015 at 7:30 PM, William Blevins <wblevins001 at gmail.com>
> wrote:
>
>>
>>
>> On Mon, Jun 8, 2015 at 4:26 PM, William Blevins <wblevins001 at gmail.com>
>> wrote:
>>
>>>
>>> On Mon, Jun 8, 2015 at 2:37 PM, Evan Driscoll <driscoll at cs.wisc.edu>
>>> wrote:
>>>
>>>> OK, so this is a bit of a weird request maybe.
>>>>
>>>> I am trying to add an SCons flag that will cause builds with GCC to use
>>>> incremental linking. This boils down to adding "-fuse-ld=gold
>>>> -fno-use-linker-plugin -Wl,--incremental" to LINKFLAGS, but I don't think
>>>> that's so relevant.
>>>>
>>>> When this flag is present, I would like to make two enhancements:
>>>> 1. Every file on which the linker is invoked should be marked as
>>>> Precious
>>>> 2. Every file on which the linker is invoked should have itself marked
>>>> as a dependency
>>>>
>>>> For my purposes, a "mostly-there" solution that only is called for
>>>> programs and not shared libraries would be fine. I also am not averse to
>>>> modifying the scons source; we have a local version with some reasonably
>>>> heavy patches anyway.
>>>>
>>>>
>>>> In case it's not clear, what I mean by #2 is the following: the output
>>>> of the linker depends not only on the .o files listed on the link line, but
>>>> on the initial state of the exe, so the has of the exe before the link
>>>> should contribute to the signature of the build step. Without some explicit
>>>> action that I'm not sure what to do, the following sequence will cause a
>>>> mis-build:
>>>>  1. Build under scons
>>>>  2. Modify an object file outside of scons
>>>>  3. Relink outside of scons
>>>>  4. Revert the object file to what it was
>>>>  5. Rebuild with scons -- it will not relink because it thinks nothing
>>>> has changed
>>>> This sequence of actions would presumably be rare, but is serious if it
>>>> happens: the result could easily be extremely hard to diagnose, and,
>>>> because we share an scons cache between different working copies and users,
>>>> a bad build could poison multiple builds. (This is enough of a concern we
>>>> would quite possibly disable caching for linked files.)
>>>>
>>>
>>> In this case, the md5sum of the resulting exe was equivalent to its
>>> previous (SCons compiled) value; otherwise, I believe SCons would rebuild
>>> the executable.
>>>
>>> I don't believe that performing incremental linking will result in a
>>> different executable md5sum versus traditional linking for shared libraries
>>> due to usage of "-fpic" (IE. PIC compliance) by default in SCons shared
>>> library building.
>>>
>>> If the md5sums differ, please explain which files clearly.
>>>
>>
>> You are right.  In the case that an item is modified, the item itself
>> will not be rebuilt.  If I make a sample SConstruct, generate a small
>> executable, and hand modify the executable, SCons still shows up-to-date
>> despite the md5sum in .sconsign.dblite being different from the actual.
>>
>> Perhaps nodes should implicitly depend on themselves.
>>
>>
>>>
>>>>
>>>
>>> I tried setting PROGEMITTER to something simple that adds the resulting
>>>> exe to the input list, but SCons complained about a circular dependency.
>>>> The main other thing that comes to mind is somewhat of a hack:
>>>>  1. For each linked file foo, make something like
>>>>       Command('foo.hash', 'foo', 'md5sum $SOURCE > $TARGET')
>>>>  2. During SConscript running, read foo.hash, put its contents into a
>>>> Value, and make foo depend on that Value
>>>> but that seems a bit ugly... and I'd still have to know how to
>>>> automatically do that for each linked file.
>>>>
>>>
>>> As long as you a using an md5sum-based decider (the SCons default), then
>>> Nodes already depend on md5sums and those sums exist in .sconsdb.sqlite
>>> file (might not have remembered the exact filename) in the same directory
>>> as your SConstruct.  You can use the sconsign scripts to get the md5sums
>>> from the database file.
>>>
>>>
>>>>
>>>> I can also imagine that a scanner could come in handy here, but that's
>>>> an area I'm basically unfamiliar with. And the man page says that it should
>>>> return a list of "File Nodes", meaning that I don't know how to get it to
>>>> return something not in the file system.
>>>>
>>>>
>>>> (Finally, just as a note: I think I haven't completely solved the
>>>> problem even with the above. Gold will embed a Build ID in the resulting
>>>> exe that will change each time, which means that with a solution to #2
>>>> SCons would always consider it out-of-date. You can specify how it
>>>> generates the Build ID, but I haven't gotten it to work quite right yet.)
>>>>
>>>> Evan
>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>>
>>> V/R,
>>> William
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20150610/d5f14fd3/attachment-0001.html>
