[Scons-users] Making SCons deal more robustly with incremental linking

[William Blevins]

On Mon, Jun 8, 2015 at 2:37 PM, Evan Driscoll <driscoll at cs.wisc.edu> wrote:

> OK, so this is a bit of a weird request maybe.
>
> I am trying to add an SCons flag that will cause builds with GCC to use
> incremental linking. This boils down to adding "-fuse-ld=gold
> -fno-use-linker-plugin -Wl,--incremental" to LINKFLAGS, but I don't think
> that's so relevant.
>
> When this flag is present, I would like to make two enhancements:
> 1. Every file on which the linker is invoked should be marked as Precious
> 2. Every file on which the linker is invoked should have itself marked as
> a dependency
>
> For my purposes, a "mostly-there" solution that only is called for
> programs and not shared libraries would be fine. I also am not averse to
> modifying the scons source; we have a local version with some reasonably
> heavy patches anyway.
>
>
> In case it's not clear, what I mean by #2 is the following: the output of
> the linker depends not only on the .o files listed on the link line, but on
> the initial state of the exe, so the has of the exe before the link should
> contribute to the signature of the build step. Without some explicit action
> that I'm not sure what to do, the following sequence will cause a mis-build:
>  1. Build under scons
>  2. Modify an object file outside of scons
>  3. Relink outside of scons
>  4. Revert the object file to what it was
>  5. Rebuild with scons -- it will not relink because it thinks nothing has
> changed
> This sequence of actions would presumably be rare, but is serious if it
> happens: the result could easily be extremely hard to diagnose, and,
> because we share an scons cache between different working copies and users,
> a bad build could poison multiple builds. (This is enough of a concern we
> would quite possibly disable caching for linked files.)
>

In this case, the md5sum of the resulting exe was equivalent to its
previous (SCons compiled) value; otherwise, I believe SCons would rebuild
the executable.

I don't believe that performing incremental linking will result in a
different executable md5sum versus traditional linking for shared libraries
due to usage of "-fpic" (IE. PIC compliance) by default in SCons shared
library building.

If the md5sums differ, please explain which files clearly.


>

I tried setting PROGEMITTER to something simple that adds the resulting exe
> to the input list, but SCons complained about a circular dependency. The
> main other thing that comes to mind is somewhat of a hack:
>  1. For each linked file foo, make something like
>       Command('foo.hash', 'foo', 'md5sum $SOURCE > $TARGET')
>  2. During SConscript running, read foo.hash, put its contents into a
> Value, and make foo depend on that Value
> but that seems a bit ugly... and I'd still have to know how to
> automatically do that for each linked file.
>

As long as you a using an md5sum-based decider (the SCons default), then
Nodes already depend on md5sums and those sums exist in .sconsdb.sqlite
file (might not have remembered the exact filename) in the same directory
as your SConstruct.  You can use the sconsign scripts to get the md5sums
from the database file.


>
> I can also imagine that a scanner could come in handy here, but that's an
> area I'm basically unfamiliar with. And the man page says that it should
> return a list of "File Nodes", meaning that I don't know how to get it to
> return something not in the file system.
>
>
> (Finally, just as a note: I think I haven't completely solved the problem
> even with the above. Gold will embed a Build ID in the resulting exe that
> will change each time, which means that with a solution to #2 SCons would
> always consider it out-of-date. You can specify how it generates the Build
> ID, but I haven't gotten it to work quite right yet.)
>
> Evan
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>

V/R,
William
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20150608/e004fc20/attachment.html>