[Scons-users] Linking a subset of common object files

[Viktor Dick]

On 2016-04-28 09:53, Ale Strooisma wrote:
> This makes me think: this is precisely the kind of problem I use a smart
> build system for! Now I realize that figuring out the dependencies for
> linking is non-trivial. I noticed however that for each object file that
> needs to be linked, the main source file depends (directly or
> indirectly) on the a header file with the same name, save for the
> extension. And a dependency tree for those header files exists. Is there
> a way to use this?

Hi,
I attached my solution, reduced to the relevant parts. Maybe you find it
helpful.

Regards,
Viktor
-------------- next part --------------
######## Initialization: set up environments and parse options #######
import os, subprocess, threading

#create the environments (with compiler flags etc.)
env = Environment()

#Now this is somewhat of a hack. The project follows the convention that if
# some function is declared in foo.h, the implementation is found in foo.cpp.
# Therefore, the list of object files that have to be linked together to create
# a specific executable can be obtained automatically. This complete tree has to be
# obtained with each call of scons, since the build tree has to be complete before the first
# element is built. Therefore, as a compromise, the scan for each needed file is done in a
# separate thread to speed things up.

#mapping of files that also have to be linked for each source file
deps = dict()
started = set() #set of files for which a scan was already started
lock = threading.Lock()

#we use 'g++ -MM' to obtain the list of .h files that are included
args=["g++","-MM", "-Isrc"]
class Scan(threading.Thread):
    def __init__(self,filename):
        threading.Thread.__init__(self)
        self.filename = filename
    def run(self):
        global started
        out = subprocess.Popen(args+[self.filename], stdout=subprocess.PIPE,universal_newlines=True).communicate()[0]
        files = [f[:-1]+"cpp" for f in out.split() if f.endswith(".h")]
        files = set([f for f in files if f != self.filename and os.path.exists(f)])
        subthreads = [] #new files that were found and will be scanned after adding them to the dict
        with lock:
            deps[self.filename] = set(files) #only level 1 dependencies, no recursion yet
            files -= started #only those that have not been started yet
            started |= files
        for f in files:
            st = Scan(f)
            st.start()
            subthreads.append(st)
        for st in subthreads:
            st.join()
            
threads = []
progs = Glob('src/main_*.cpp', strings=True)

# start scan with main files, recursing further down
for f in progs:
    t = Scan(f)
    threads.append(t)
    t.start()
for t in threads:
    t.join()

# now collect everything that needs to be linked for some target
def collect(f):
    result = set()
    todo = set([f])
    while len(todo)>0:
        e = todo.pop()
        if e in deps and e not in result:
            todo.update(deps[e])
        result.add(e)
    return list(result)


for f in progs:
    b = 'bin/'+os.path.basename(f)[5:-4]
    env.Program(b, collect(f))
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20160428/8be0a7a2/attachment.pgp>