[Scons-users] Thin archive Tool

[Bill Deegan]

I'm guessing that using archives let's him skip keeping track of which
object files need to be pulled in (as opposed to listing all the object
files).

-Bill

On Fri, Oct 21, 2016 at 11:26 AM, Rob Boehne <robb at datalogics.com> wrote:

> Andrew,
>
> Just out of curiosity, why are you using archives at all?  There are
> legitimate reasons, and I haven’t looked at MongoDB, but perhaps removing
> this step would simplify your build and solve this issue.
>
> HTH,
>
> Robert Boehne
>
> From: Scons-users <scons-users-bounces at scons.org> on behalf of "Andrew C.
> Morrow" <andrew.c.morrow at gmail.com>
> Reply-To: SCons users mailing list <scons-users at scons.org>
> Date: Thursday, October 20, 2016 at 3:32 PM
> To: SCons users mailing list <scons-users at scons.org>
> Subject: [Scons-users] Thin archive Tool
>
>
> Hi -
>
> The SCons based build system for MongoDB makes heavy use of static
> linking. One consequence of static linking is that the space requirements
> are basically doubled, since each translation unit produces an object file,
> and then each object file is copied into an archive file. Adding CacheDir
> into the mix multiplies this duplication.
>
> The GNU binutils tools, however, support 'thin' archives, where the
> archive contents are simply a list of file references, meaning that the
> archive files are very small. At link time, the linker simply dereferences
> the listed files in each archive.
>
> To support this, we added the following Apache 2.0 licensed tool:
>
> https://github.com/mongodb/mongo/blob/master/site_scons/
> site_tools/thin_archive.py
>
> One subtle aspect to consider is that when using thin archives, if
> application X depends on libY.a, which contains z.o produced from z.c, then
> if z.c is changed, the built-in signature of libY.a will not change, since
> the reference to z.o doesn't change when the archive is rebuilt, so taking
> the MD5 of the file contents will yield the same result.
>
> To address this, our Tool creates a new Node subclass that overrides the
> get_contents and get_content_hash methods, and sets the target_factory for
> StaticLibrary to produce that Node subclass. The overriding behavior
> computes a content hash based on the content hash of the children of the
> new node.
>
> This all seems to work, fairly well, but I was curious if there was a more
> appropriate way to accomplish this. The end goal is that we want the
> content signature of these Nodes to be a hash of the signatures of all of
> the Nodes children, rather than the on-disk contents of the Node.
>
> Is there a better way to accomplish this than what we are doing here?
>
> FWIW we also use a similar technique for driving ABI change linking when
> doing dynamic builds:
>
> https://github.com/mongodb/mongo/blob/master/site_scons/
> site_tools/abilink.py
>
> Here, we would be particularly interested in arranging to interpose in
> such a way to absolutely minimize the number of times we must invoke abidw,
> as that is very expensive. Avoiding needless re-invocations of abidw as
> things move in and out of the CacheDir is desired.
>
> Thanks,
> Andrew
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20161021/80ad70a5/attachment.html>