[Scons-users] Make all Program and SharedLibrary targets implicitly Precious?

Tue May 17 04:47:47 EDT 2016

Hi Andrew,

I actually needed to do LINKFLAGS='-Wl,-fuse-ld=gold' and I must say it
gives an insane speed-up. I am really happy I found out about this tool!

Kind regards,
Ale Strooisma

On 13 May 2016 at 15:47, Andrew C. Morrow <andrew.c.morrow at gmail.com> wrote:

>
> Hi -
>
> The equivalent of env.Append(LINKFLAGS=['-fuse-ld=gold']) is what MongoDB
> uses to enable the gold linker:
>
>
> https://github.com/mongodb/mongo/blob/d9a8e6a9db1bd48c2bbfb1ad18e7cb8e18eb7302/SConstruct#L2164-L2167
>
> You almost certainly don't want to change LINK in a C++ build, for the
> reasons you found. SCons is really fairly smart about how to invoke the
> linker correctly (search for SmartLink).
>
> Hope this helps,
> Andrew
>
>
> On Fri, May 13, 2016 at 4:56 AM, Ale Strooisma <
> a.strooisma at student.utwente.nl> wrote:
>
>> Ah okay, that works, thanks! Silly that I did not come up with that
>> myself.
>> Still a few more libraries to add, though. I'm definitely going to try
>> your alternative.
>>
>> Kind regards,
>> Ale Strooisma
>>
>> On 13 May 2016 at 10:44, Jörg Plate <patterner at gmail.com> wrote:
>>
>>> The stdc++ library tends to be in non-standard locations, since it's
>>> depending on the compiler.
>>> Try
>>>   locate "*libstdc++*so*"
>>> and use the -L option with the correct directory. My g++ uses
>>> /usr/lib/gcc/x86_64-linux-gnu/6/.
>>>
>>> Alternative:   use g++ with "-Wl,-fuse-ld=gold" and let g++ deal with it.
>>>
>>> On Fri, May 13, 2016 at 10:30 AM, Ale Strooisma
>>> <a.strooisma at student.utwente.nl> wrote:
>>> > Andrew,
>>> >
>>> > inspired by this discussion I decided to try using gold as well. I ran
>>> into
>>> > a problem however, and can't seem to find a lot of documentation on
>>> gold, so
>>> > I hoped you could shed some light on this problem as you are working
>>> with
>>> > gold and scons.
>>> >
>>> > I changed to gold by adding LINK='ld.gold' to my environment object.
>>> > However, now I get a lot of undefined reference errors during the
>>> linking
>>> > process - they all seem to come from the standard libraries. I
>>> interpret
>>> > this as gold not linking those by default, so I added stdc++ to LIBS,
>>> but
>>> > that resulted in "ld.gold: error: cannot find -lstdc++".
>>> >
>>> > Any tips for getting this to work?
>>> >
>>> > Kind regards,
>>> > Ale Strooisma
>>> >
>>> >
>>> > On 13 May 2016 at 00:28, Bill Deegan <bill at baddogconsulting.com>
>>> wrote:
>>> >>
>>> >> Andrew,
>>> >>
>>> >> Looks reasonable.
>>> >> What happens to the old binary if you os.remove it on one system, when
>>> >> it's being used on another? (NFS)
>>> >> Does it crash, or is the file kept alive via inode reference by
>>> running
>>> >> process?
>>> >>
>>> >> -Bill
>>> >>
>>> >> On Thu, May 12, 2016 at 4:08 PM, Andrew C. Morrow
>>> >> <andrew.c.morrow at gmail.com> wrote:
>>> >>>
>>> >>>
>>> >>> OK, I got something that sort of works for incremental linking. I'm
>>> sure
>>> >>> it can be improved:
>>> >>>
>>> >>>
>>> >>>
>>> https://github.com/acmorrow/mongo/blob/0062b8bfa0c24812438f4765fa20755fb5f6fc54/site_scons/site_tools/incremental_link.py
>>> >>>
>>> >>> The binaries that ld.gold produces seem terribly broken, but perhaps
>>> it
>>> >>> will be useful on Windows.
>>> >>>
>>> >>> I wonder if .Precious should have a mode where it behaves like this
>>> by
>>> >>> default, or if it should take an optional parameter. It seems useful.
>>> >>>
>>> >>> On Thu, May 12, 2016 at 12:18 PM, Andrew C. Morrow
>>> >>> <andrew.c.morrow at gmail.com> wrote:
>>> >>>>
>>> >>>>
>>> >>>> Hi Bill -
>>> >>>>
>>> >>>> I like this suggestion of adding a PreAction to each executable and
>>> >>>> shared library, but I don't want to have to explicitly call
>>> .AddPreAction on
>>> >>>> each target.
>>> >>>>
>>> >>>> Can you suggest an easy way to achieve the same effect globally for
>>> all
>>> >>>> SharedLibrary and Program targets, without writing a PseudoBuilder?
>>> I'm fine
>>> >>>> though with reaching into the existing builders and altering them.
>>> >>>>
>>> >>>> Thanks,
>>> >>>> Andrew
>>> >>>>
>>> >>>> On Sat, Apr 30, 2016 at 4:38 PM, Bill Deegan <
>>> bill at baddogconsulting.com>
>>> >>>> wrote:
>>> >>>>>
>>> >>>>> Andrew,
>>> >>>>>
>>> >>>>> How about a pre-action to mv the existing file to another name, and
>>> >>>>> then copy it to original name, then incremental link the copy?
>>> >>>>> That should resolve the crashing bit.
>>> >>>>>
>>> >>>>> lsof filename won't help if the files being used on another
>>> system, but
>>> >>>>> would if on current system. (to see if in use)
>>> >>>>>
>>> >>>>> -Bill
>>> >>>>>
>>> >>>>> On Sat, Apr 30, 2016 at 3:47 PM, Andrew C. Morrow
>>> >>>>> <andrew.c.morrow at gmail.com> wrote:
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> Definitely a possibility. MongoDB probes for compiler support for
>>> >>>>>> ld.gold and prefers it by default:
>>> >>>>>>
>>> >>>>>>
>>> https://github.com/mongodb/mongo/blob/master/SConstruct#L2164-L2167
>>> >>>>>>
>>> >>>>>> The downside to using incremental linking and making dynamic
>>> targets
>>> >>>>>> Precious is that by overwriting the binaries you end up crashing
>>> running
>>> >>>>>> programs that have them mapped. That happens often enough during
>>> developer
>>> >>>>>> workflow that it is unfortunate.
>>> >>>>>>
>>> >>>>>> So, it definitely should be opt-in behavior, and a Tool sounds
>>> right
>>> >>>>>> for that.
>>> >>>>>>
>>> >>>>>> I also wonder whether such a Tool might be able to make
>>> intelligent
>>> >>>>>> use of gold's --incremental-base file to avoid the above problem.
>>> If you
>>> >>>>>> first copied the dynamic target aside under a different name,
>>> then unlinked
>>> >>>>>> the original, then used --incremental-base pointing to the
>>> moved-aside file,
>>> >>>>>> perhaps you could avoid crashing programs that have the target
>>> mapped. I'm
>>> >>>>>> not sure though if the cost of the file copy would eat up most of
>>> the gains
>>> >>>>>> from incremental linking.
>>> >>>>>>
>>> >>>>>> That sounds complex enough to also warrant a separate Tool.
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> On Fri, Apr 29, 2016 at 1:52 PM, Bill Deegan
>>> >>>>>> <bill at baddogconsulting.com> wrote:
>>> >>>>>>>
>>> >>>>>>> Andrew,
>>> >>>>>>>
>>> >>>>>>> Might make sense to make a gold linker tool?
>>> >>>>>>>
>>> >>>>>>> -Bill
>>> >>>>>>>
>>> >>>>>>> On Fri, Apr 29, 2016 at 9:47 AM, Andrew C. Morrow
>>> >>>>>>> <andrew.c.morrow at gmail.com> wrote:
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> Thanks Bill -
>>> >>>>>>>>
>>> >>>>>>>> I think that sounds promising, I'll give it a look. If I get it
>>> >>>>>>>> working well I'll ping the list back and describe my findings.
>>> >>>>>>>>
>>> >>>>>>>> Thanks,
>>> >>>>>>>> Andrew
>>> >>>>>>>>
>>> >>>>>>>> On Fri, Apr 29, 2016 at 12:10 PM, Bill Deegan
>>> >>>>>>>> <bill at baddogconsulting.com> wrote:
>>> >>>>>>>>>
>>> >>>>>>>>> Andrew,
>>> >>>>>>>>>
>>> >>>>>>>>> Likely you want to stick this logic in the emitter for building
>>> >>>>>>>>> libraries and programs.
>>> >>>>>>>>> SCons/Tool/link.py  shlib_emitter() is the current logic.
>>> >>>>>>>>> The environment tracks this in:
>>> >>>>>>>>> env['SHLIBEMITTER'] which is a list.
>>> >>>>>>>>> So if you append to SHLIBEMITTER and mark the relevant nodes
>>> >>>>>>>>> Precious()
>>> >>>>>>>>> env['PROGEMITTER'] is the emitter for programs. Looks like it's
>>> >>>>>>>>> only defined for mslink and qt, so you can defined one on non
>>> qt/mslink
>>> >>>>>>>>> freely and do the same.
>>> >>>>>>>>>
>>> >>>>>>>>> Hope that helps
>>> >>>>>>>>> -Bill
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> On Fri, Apr 29, 2016 at 8:52 AM, William Blevins
>>> >>>>>>>>> <wblevins001 at gmail.com> wrote:
>>> >>>>>>>>>>
>>> >>>>>>>>>> Andrew,
>>> >>>>>>>>>>
>>> >>>>>>>>>> I expect that you could make a Pseudo-Builder that wraps
>>> Program
>>> >>>>>>>>>> and Library calls, and replace the top-level calls with your
>>> wrappers.
>>> >>>>>>>>>>
>>> >>>>>>>>>> I think you can replace them like env['BUILDERS']['xxxxxx'] =
>>> >>>>>>>>>> Wrapper or env.AddMethod but I haven't tested this explicitly.
>>> >>>>>>>>>>
>>> >>>>>>>>>> https://bitbucket.org/scons/scons/wiki/ToolsForFools
>>> >>>>>>>>>>
>>> >>>>>>>>>> V/R,
>>> >>>>>>>>>> William
>>> >>>>>>>>>>
>>> >>>>>>>>>> On Fri, Apr 29, 2016 at 4:21 PM, Andrew C. Morrow
>>> >>>>>>>>>> <andrew.c.morrow at gmail.com> wrote:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Hi -
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Is there a simple way to do this? I'd like to use ld.gold's
>>> >>>>>>>>>>> incremental linking, but I don't want to need to add
>>> .Precious to every
>>> >>>>>>>>>>> executable and library in my project.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Thanks,
>>> >>>>>>>>>>> Andrew
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> _______________________________________________
>>> >>>>>>>>>>> Scons-users mailing list
>>> >>>>>>>>>>> Scons-users at scons.org
>>> >>>>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>> >>>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> _______________________________________________
>>> >>>>>>>>>> Scons-users mailing list
>>> >>>>>>>>>> Scons-users at scons.org
>>> >>>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> _______________________________________________
>>> >>>>>>>>> Scons-users mailing list
>>> >>>>>>>>> Scons-users at scons.org
>>> >>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>> >>>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> _______________________________________________
>>> >>>>>>>> Scons-users mailing list
>>> >>>>>>>> Scons-users at scons.org
>>> >>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>> >>>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> _______________________________________________
>>> >>>>>>> Scons-users mailing list
>>> >>>>>>> Scons-users at scons.org
>>> >>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>> >>>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> _______________________________________________
>>> >>>>>> Scons-users mailing list
>>> >>>>>> Scons-users at scons.org
>>> >>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>> >>>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> _______________________________________________
>>> >>>>> Scons-users mailing list
>>> >>>>> Scons-users at scons.org
>>> >>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>> >>>>>
>>> >>>>
>>> >>>
>>> >>>
>>> >>> _______________________________________________
>>> >>> Scons-users mailing list
>>> >>> Scons-users at scons.org
>>> >>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>> >>>
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> Scons-users mailing list
>>> >> Scons-users at scons.org
>>> >> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>> >>
>>> >
>>> >
>>> > _______________________________________________
>>> > Scons-users mailing list
>>> > Scons-users at scons.org
>>> > https://pairlist4.pair.net/mailman/listinfo/scons-users
>>> >
>>>
>>>
>>>
>>> --
>>> "I'm working on it."
>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20160517/0e49ec4b/attachment-0001.html>