[Scons-users] CompilationDatabase support for SCons

[Bill Deegan]

Andrew,

Have you had the developers use SCons's interactive mode for quick
re-builds?

-Bill

On Sun, Jan 11, 2015 at 3:27 PM, Andrew C. Morrow <andrew.c.morrow at gmail.com
> wrote:

>
>
> On Sun, Jan 11, 2015 at 12:33 PM, Gary Oberbrunner <garyo at oberbrunner.com>
> wrote:
>
>>
>>
>> On Sun, Jan 11, 2015 at 11:27 AM, Andrew C. Morrow <
>> andrew.c.morrow at gmail.com> wrote:
>>
>>>
>>> Hi Gary -
>>>
>>> Thanks for getting back to me.
>>>
>>> The utility of a compilation database
>>> <http://clang.llvm.org/docs/JSONCompilationDatabase.html> is that it
>>> lets you capture with fidelity the compile actions that the build system
>>> would take (given the current options, configure checks, etc), and then
>>> load that information into other tools. A good overview of why this is
>>> needed can be found here:
>>> http://eli.thegreenplace.net/2014/05/21/compilation-databases-for-clang-based-tools
>>>
>>> OK, from that I gather that it's a clang-specific database of which
>> flags were used to compile which source files -- presumably from the last
>> build.
>>
>
> Correct. That it is clang specific is why I would prefer to do this by
> building a SCons Tool, rather than needing to build support for generating
> it into SCons itself. FWIW, CMake and Ninja seem to have baked support
> directly into the build system though.
>
>
>>
>>
>>> For some additional practical examples, here are some examples of things
>>> that can be done once you have a compilation database (usually named
>>> compile_commands.json):
>>>
>>> - Run clang-modernize <http://clang.llvm.org/extra/clang-modernize.html> (convert
>>> C++03 to C++11) and clang-tidy
>>> <http://clang.llvm.org/extra/clang-tidy.html> (AST aware linter)
>>> - Configure emacs or VI to have a "compile current buffer" facility, by
>>> extracting the last compile command for the current buffer (
>>> https://github.com/randomphrase/ede-compdb)
>>>
>>
>> This in particular seems like a bad idea.  If you tell SCons to recompile
>> the current buffer (its .o file) it will check for any modified headers,
>> different compile flags, generated sources that need generating, and so on,
>> before issuing the compile command -- and that only if needed.  Re-running
>> the same last command seems at best error-prone, and at worst it could
>> confuse SCons because it doesn't know you updated the .o outside of it (so
>> it doesn't get into the repository if you're using one, and so on).
>>
>
> Well, I'll say that from having worked on several codebases using SCons,
> developers frequently copy and paste compiler invocation lines into the
> shell and run them, and then run them again and again via Ctrl-R. The most
> frequent reason I've heard for doing this is that it takes too long for the
> build system to start up.
>
> In any event, this is just one use case out of dozens, so even if it isn't
> a compelling example, I don't think it alone invalidates the idea of
> producing a compile_commands.json file.
>
>
>>
>> - Drive your own AST level source introspection or transformation tools
>>> implemented with clang's LibTooling
>>> <http://clang.llvm.org/docs/LibTooling.html>
>>> - Simplify configuration of emacs flycheck or VI you-complete-me tools
>>> for real-time in-editor syntax checking
>>>
>>> I'm sure there are many more I'm not aware of.
>>>
>>> I acknowledge that this is all fairly clang specific, which is why my
>>> hope had been that it would be possible to generate a compile_commands.json
>>> file in the build system (by writing a SCons Tool), rather than by the
>>> build system (needing to implement it as an intrinsic SCons feature, as I
>>> believe CMake and Ninja do).
>>>
>>> Also, I think it is very important that it be possible to produce the
>>> compilation database without needing to actually compile the entire source
>>> tree. For some projects a complete rebuild may be prohibitively expensive,
>>> but this should not preclude using the above tools. My second attempt above
>>> does work, but has the unwanted side effect that all objects are built, so
>>> does not meet this goal.
>>>
>>
>> I'm pretty sure this can't work in the general case.  It's easy with
>> SCons to create generated headers and sources, for instance, and even
>> introspect sources (which may be generated) to see what needs to be
>> compiled, and how.  (My own day-job build does exactly this, by building a
>> precompiler (written in C++), precompiling a proprietary high-level
>> language into C++ code which is then compiled.) You can't just do an
>> old-school "make -n" and have it be likely to resemble the real list of
>> commands, except in very simple, straightforward cases.
>>
>
> Whether or not it works in the general case is a separate issue. If it
> doesn't work for you, don't use it. The clang tooling ecosystem is widely
> used in practice, and those tools are driven by compilation databases
> produced by build systems like CMake and Ninja. Obviously, there are build
> environments where it is not a viable technique. But surely that isn't an
> argument against making it work with SCons in the cases where it can and
> enables access to a set of very powerful new development tools?
>
>
>>
>> In a simple C++-only single-pass build, I can see this working to some
>> extent.  I suspect the simplest method is to wrap your compiler with a
>> shell script that emits $* to the database before (or after?) running the
>> comand, and use that shell script as $CC/$CXX etc.  Modifying the existing
>> tools as you've started to do could also work, as could overriding the
>> CC*OMSTR variables perhaps.  However, these will only produce the partial
>> results of what actually got compiled -- if you change flags or compiler
>> versions or paths, you could end up with an out-of-sync database.
>>
>
> Right, and this is why I want to do it as something that SCons does
> understand. The shell script approach is not viable for day to day
> development, and as you point out is unlikely to give complete results and
> has synchronization and correctness issues.
>
> Having SCons produce the file as a consequence of walking the graph
> ensures that the genereated compile_commands.json file does reflect reality
> and won't be out of date.
>
> Anyway, thanks for your thoughts on the matter.
>
> Andrew
>
>
>
>>
>> --
>> Gary
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20150111/aa4f658d/attachment-0001.html>