[Scons-users] CompilationDatabase support for SCons

[Andrew C. Morrow]

On Sun, Jan 11, 2015 at 12:33 PM, Gary Oberbrunner <garyo at oberbrunner.com>
wrote:

>
>
> On Sun, Jan 11, 2015 at 11:27 AM, Andrew C. Morrow <
> andrew.c.morrow at gmail.com> wrote:
>
>>
>> Hi Gary -
>>
>> Thanks for getting back to me.
>>
>> The utility of a compilation database
>> <http://clang.llvm.org/docs/JSONCompilationDatabase.html> is that it
>> lets you capture with fidelity the compile actions that the build system
>> would take (given the current options, configure checks, etc), and then
>> load that information into other tools. A good overview of why this is
>> needed can be found here:
>> http://eli.thegreenplace.net/2014/05/21/compilation-databases-for-clang-based-tools
>>
>> OK, from that I gather that it's a clang-specific database of which flags
> were used to compile which source files -- presumably from the last build.
>

Correct. That it is clang specific is why I would prefer to do this by
building a SCons Tool, rather than needing to build support for generating
it into SCons itself. FWIW, CMake and Ninja seem to have baked support
directly into the build system though.


>
>
>> For some additional practical examples, here are some examples of things
>> that can be done once you have a compilation database (usually named
>> compile_commands.json):
>>
>> - Run clang-modernize <http://clang.llvm.org/extra/clang-modernize.html> (convert
>> C++03 to C++11) and clang-tidy
>> <http://clang.llvm.org/extra/clang-tidy.html> (AST aware linter)
>> - Configure emacs or VI to have a "compile current buffer" facility, by
>> extracting the last compile command for the current buffer (
>> https://github.com/randomphrase/ede-compdb)
>>
>
> This in particular seems like a bad idea.  If you tell SCons to recompile
> the current buffer (its .o file) it will check for any modified headers,
> different compile flags, generated sources that need generating, and so on,
> before issuing the compile command -- and that only if needed.  Re-running
> the same last command seems at best error-prone, and at worst it could
> confuse SCons because it doesn't know you updated the .o outside of it (so
> it doesn't get into the repository if you're using one, and so on).
>

Well, I'll say that from having worked on several codebases using SCons,
developers frequently copy and paste compiler invocation lines into the
shell and run them, and then run them again and again via Ctrl-R. The most
frequent reason I've heard for doing this is that it takes too long for the
build system to start up.

In any event, this is just one use case out of dozens, so even if it isn't
a compelling example, I don't think it alone invalidates the idea of
producing a compile_commands.json file.


>
> - Drive your own AST level source introspection or transformation tools
>> implemented with clang's LibTooling
>> <http://clang.llvm.org/docs/LibTooling.html>
>> - Simplify configuration of emacs flycheck or VI you-complete-me tools
>> for real-time in-editor syntax checking
>>
>> I'm sure there are many more I'm not aware of.
>>
>> I acknowledge that this is all fairly clang specific, which is why my
>> hope had been that it would be possible to generate a compile_commands.json
>> file in the build system (by writing a SCons Tool), rather than by the
>> build system (needing to implement it as an intrinsic SCons feature, as I
>> believe CMake and Ninja do).
>>
>> Also, I think it is very important that it be possible to produce the
>> compilation database without needing to actually compile the entire source
>> tree. For some projects a complete rebuild may be prohibitively expensive,
>> but this should not preclude using the above tools. My second attempt above
>> does work, but has the unwanted side effect that all objects are built, so
>> does not meet this goal.
>>
>
> I'm pretty sure this can't work in the general case.  It's easy with SCons
> to create generated headers and sources, for instance, and even introspect
> sources (which may be generated) to see what needs to be compiled, and how.
>  (My own day-job build does exactly this, by building a precompiler
> (written in C++), precompiling a proprietary high-level language into C++
> code which is then compiled.) You can't just do an old-school "make -n" and
> have it be likely to resemble the real list of commands, except in very
> simple, straightforward cases.
>

Whether or not it works in the general case is a separate issue. If it
doesn't work for you, don't use it. The clang tooling ecosystem is widely
used in practice, and those tools are driven by compilation databases
produced by build systems like CMake and Ninja. Obviously, there are build
environments where it is not a viable technique. But surely that isn't an
argument against making it work with SCons in the cases where it can and
enables access to a set of very powerful new development tools?


>
> In a simple C++-only single-pass build, I can see this working to some
> extent.  I suspect the simplest method is to wrap your compiler with a
> shell script that emits $* to the database before (or after?) running the
> comand, and use that shell script as $CC/$CXX etc.  Modifying the existing
> tools as you've started to do could also work, as could overriding the
> CC*OMSTR variables perhaps.  However, these will only produce the partial
> results of what actually got compiled -- if you change flags or compiler
> versions or paths, you could end up with an out-of-sync database.
>

Right, and this is why I want to do it as something that SCons does
understand. The shell script approach is not viable for day to day
development, and as you point out is unlikely to give complete results and
has synchronization and correctness issues.

Having SCons produce the file as a consequence of walking the graph ensures
that the genereated compile_commands.json file does reflect reality and
won't be out of date.

Anyway, thanks for your thoughts on the matter.

Andrew



>
> --
> Gary
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20150111/99ace7dd/attachment-0001.html>