[Scons-users] Three enhancement ideas

[Andrew C. Morrow]

Based on recent build system work at MongoDB, we have the following three
ideas for potential new features for SCons. All three are somewhat
speculative and come with some open questions. Other suggestions for how to
approach the indicated problems are also welcome. In keeping with the
guidance to bring ideas to the user group before filing issues, here they
are:


- Automatic "generated sources" list.

One of the MongoDB tools is the compilation database tool, which builds a
compilation database for a project without building the sources. The fact
that we don't build the sources is important - it takes a long time to
build all the sources, but engineers often want a compilation database
right away so they can use it as part of an IDE, or so they can run tooling
like clang-tidy on specific files without needing to wait for a full build
first.

However, often there are a small number of generated source files, like a
config,h header, that are required in order to have a tool like clang-tidy
work. Without them, the compiler front end can't find necessary include
files.

To solve this, MongoDB has an Alias called `generated-sources`, to which we
manually add every generated source. This allows engineers to just build
the set of generated sources required to make compilation database tooling
work. However, remembering to add to the Alias is inconvenient and easily
overlooked, so it often breaks.

We are wondering whether it may be possible for SCons itself to identify
and automatically track generated sources, and then expose them via a
method similar to the `FindInstalledFiles` method. Such a facility would
also be useful in our Ninja backend for SCons, which also needs to know
about generated sources.

There are definitely some open questions: What makes something a generated
"source"? How can they be identified? Would SCons be able to
deterministically identify such and expose them? How does the information
get scoped? Per Environment? globally?

I'd be interested in your thoughts on this, particularly whether it is
worth opening this as an enhancement request on GH.


- Programmatically expose all SCons* files used in the build.

This request is driven by work on our Ninja builder for SCons. The Ninja
builder needs to declare that the generated build.ninja file have a
dependency on SConstruct and all the SConscripts, so that we can detect
when the build.ninja file should be regenerated, because the structure of
the build as described by SCons has changed.

Currently, we do this by globbing for such files, but this is not a great
long term solution. In particular, env.SConscript allows the scripts to be
named something other than SConscript, so if we want our Ninja tool to be
truly generic, it shouldn't rely on filenames and globbing to do this work.

There are some tricky questions with this one, specifically, with how the
phases of SCons execution would populate this vs when the Ninja backend
would need access to the list.

Still, if there is still general interest in moving forward with getting
the MongoDB Ninja generator upstreamed into SCons, some more robust
solution to identifying the SCons files in play will be needed.

As above, interested in thoughts, and whether a GH issue seems worth
opening.


- Programmatic identification of conftest nodes.

Much like the above, several of our tools work by interposing on emitters
or scanners of existing builders. In some cases, the work we want to
perform is inappropriate for conftests, either because of the special
environment in which they execute, or because the injected behavior is
inappropriate for conftests.

For now, we need to do something ugly like

```
if not "conftest" in str(target[0):
    # Do stuff to non-conftest nodes
```

which is unpleasant and far from foolproof. We think it would be better if
there were an attribute or method on Node that would allow us to identify
conftest-related nodes directly. Ideally, we could write something better,
like:

```
if not target[0].is_conftest():
    # Do stuff to non-conftest nodes
```

Alternatively:

```
if not getattr(target[0].attributes, "is_scons_conftest", False):
    # Do stuff to non-conftest nodes
```

Would it be possible to adorn all nodes used by the SConf subsystem this
way? It would make it much easier to handle such nodes differently  when
required, without needing to worry that someone is going to introduce a
file called "conftest.cpp" next year when they want to write a test for
some configuration, not knowing that that name has special meaning to SCons
and our tools.

As above, I'm interested in thoughts and comments, and whether a GH issue
is OK for this.



Thanks,
Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20200422/6d091a0c/attachment.html>