[Scons-users] Source paths across external projects

[Bobby Casey]

Hello,

I'm going to start by apologizing for the length of this email.  I started
off trying to explain what I'm looking for and what I have done and ended
up trying other ideas as I went along.

So here's what I'm trying to do - I have a project that depends on several
external source repositories. The structure looks something like this:

top_project/
---> SConstruct
---> SConscript
---> sources.scons
---> extern/foo
------> SConstruct
------> SConscript
------> sources.scons
------> src/
---------> foo.c
---> extern/bar
......

Where each of the "extern" projects (there are several) is a separate
repository that is capable of building itself (generally for testing
purposes) and contains a sources.scons file that top_project's SConscript
file pulls in to grab the list of source files that the sub-project
provides. The problem, however, is that when `top_project/SConscript' calls
`SConscript("extern/foo/sources.scons")', all the paths are relative to
top_project (rather than extern/foo).

This is a bit hard to explain, let me give a visual.

In the example above, `top_project/SConscript' would look like this:
    src = SConscript('sources.scons')
    foo =SConscript('extern/foo/sources.scons')
    src.extend(foo)
    prog = Program('test', src)

and `extern/foo/sources.scons' would look like this:
    src = ['src/foo.c']
    Return('src')

This will fail with the following error:
    scons: *** [build/src/foo.o] Source `build/src/foo.c' not found, needed
by target `build/src/foo.o'.

That is, of course, the correct behavior - just not the behavior what I
wanted!  I could prepend os.path.abspath before each file, but that leaves
the object files next to the sources, ignoring any variant_dir that was
specified in top_project/SConstruct.

I worked around this by modifying top_project/SConscript to look like this
instead:
    def add_scons_source(proj_dir):
        src = SConscript('%s/sources.scons' % proj_dir)
        src = ['%s/%s' % (proj_dir, s) for s in src]
        return src

    src = SConscript('sources.scons')
    src.extend( add_scons_source('extern/foo') )
    prog = Program('test', src)

This prepends the appropriate path to each source file and works just fine
but feels decidedly "unsconsy".  I then considered having each
sources.scons be responsible for building it's own object files and that
works great.  It's exactly what I wanted for building the application.  I
suspect this is also the canonically correct way to do it.  The problem is
that, in our real application, we also have a lint target that needs to be
run against all source files so that it can detect issues across
compilation units.  I'd like to avoid having the external projects know
anything about lint, or whatever other targets are available in the
top_project, so I came up with this as a possible solution.

top_project/SConscript would look something like this:
    env = Environment()
    source_list = []
    def do_lint(target, source, env):
        print 'running lint on %s' % (str(s) for s in source)

    def aggregate_sources(target, source, env):
        global source_list
        source_list.extend( [str(s) for s in source] )

    env.Append(BUILDERS={'AggregateSources':
Builder(action=aggregate_sources)})
    env.Append(BUILDERS={'LintBuilder': Builder(action=do_lint)})

    SrcBuilder = env.Object
    src = SConscript('sources.scons', exports=['SrcBuilder'])
    src.extend( SConscript('extern/foo/sources.scons',
exports=['SrcBuilder']) )

    prog = env.Program('test', src)
    Default(prog)

    SrcBuilder = env.AggregateSources
    lint_src = SConscript('sources.scons', exports=['SrcBuilder'])
    lint_src.extend( SConscript('extern/foo/sources.scons',
exports=['SrcBuilder']) )
    lint = env.LintBuilder('lint.out', lint_src)

    Alias('lint', lint)

and the sources.scons files would all look something like this:

    Import('SrcBuilder')
    src = ['src/foo.c']
    built = [SrcBuilder(s) for s in src]
    Return('built')

This looks a bit silly in this contrived example because the SrcBuilder
would really be part of two separate environments (one for Program and one
for Lint targets), but I think you get the point.  The advantage to this
method is that the builder is provided to sources.scons so it need not know
how we're using the sources but can still handle them appropriately.  The
disadvantage is that SrcBuilder needs to either be some common name for two
wildly different things that are either part of the environment or passed
as an extern.  It would be nicer if I could say something like `export(
env.Object as "SrcBuilder")' and `export( env.AggregateSources as
"SrcBuilder")'.  Is there some way to do this?

I would appreciate any comments/feedback/suggestions/criticism.  Are any of
these methods even remotely sound?  Is there some canonical way of doing
this that I have just totally missed somehow?  Am I proposing a flawed
architecture that could be easier/better done in some other way?

Thanks in advance,
  Bobby
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20160908/28bdd4da/attachment.html>