[Scons-users] CompilationDatabase support for SCons

Sun Jan 11 18:45:12 EST 2015

Andrew,

Maybe I don't fully understand what you are trying to accomplish, but I
find that it's easy to ask the wrong questions.  I still don't think you
have explained what you are trying to accomplish which isn't the same as
what you have tried and decided didn't work.

If you just want "What SCons commands SCons would run if SCons users could
run SCons?" Then it might be straight-forward enough to pipe "SCons
--dry-run" into a file or another script.  You don't have to wait for code
to compile and you get the most up-to-date commands.

Well, I'll say that from having worked on several codebases using SCons,
> developers frequently copy and paste compiler invocation lines into the
> shell and run them, and then run them again and again via Ctrl-R. The most
> frequent reason I've heard for doing this is that it takes too long for the
> build system to start up.

IMPE: most user's with this compliant don't understand how to call SCons on
specific targets, and the others would rather risk losing hours debugging
an incomplete rebuild than wait the 1 second to read the SConscripts.

Maybe this was helpful,
William

On Sun, Jan 11, 2015 at 6:27 PM, Andrew C. Morrow <andrew.c.morrow at gmail.com
> wrote:

>
>
> On Sun, Jan 11, 2015 at 12:33 PM, Gary Oberbrunner <garyo at oberbrunner.com>
> wrote:
>
>>
>>
>> On Sun, Jan 11, 2015 at 11:27 AM, Andrew C. Morrow <
>> andrew.c.morrow at gmail.com> wrote:
>>
>>>
>>> Hi Gary -
>>>
>>> Thanks for getting back to me.
>>>
>>> The utility of a compilation database
>>> <http://clang.llvm.org/docs/JSONCompilationDatabase.html> is that it
>>> lets you capture with fidelity the compile actions that the build system
>>> would take (given the current options, configure checks, etc), and then
>>> load that information into other tools. A good overview of why this is
>>> needed can be found here:
>>> http://eli.thegreenplace.net/2014/05/21/compilation-databases-for-clang-based-tools
>>>
>>> OK, from that I gather that it's a clang-specific database of which
>> flags were used to compile which source files -- presumably from the last
>> build.
>>
>
> Correct. That it is clang specific is why I would prefer to do this by
> building a SCons Tool, rather than needing to build support for generating
> it into SCons itself. FWIW, CMake and Ninja seem to have baked support
> directly into the build system though.
>
>
>>
>>
>>> For some additional practical examples, here are some examples of things
>>> that can be done once you have a compilation database (usually named
>>> compile_commands.json):
>>>
>>> - Run clang-modernize <http://clang.llvm.org/extra/clang-modernize.html> (convert
>>> C++03 to C++11) and clang-tidy
>>> <http://clang.llvm.org/extra/clang-tidy.html> (AST aware linter)
>>> - Configure emacs or VI to have a "compile current buffer" facility, by
>>> extracting the last compile command for the current buffer (
>>> https://github.com/randomphrase/ede-compdb)
>>>
>>
>> This in particular seems like a bad idea.  If you tell SCons to recompile
>> the current buffer (its .o file) it will check for any modified headers,
>> different compile flags, generated sources that need generating, and so on,
>> before issuing the compile command -- and that only if needed.  Re-running
>> the same last command seems at best error-prone, and at worst it could
>> confuse SCons because it doesn't know you updated the .o outside of it (so
>> it doesn't get into the repository if you're using one, and so on).
>>
>
> Well, I'll say that from having worked on several codebases using SCons,
> developers frequently copy and paste compiler invocation lines into the
> shell and run them, and then run them again and again via Ctrl-R. The most
> frequent reason I've heard for doing this is that it takes too long for the
> build system to start up.
>
> In any event, this is just one use case out of dozens, so even if it isn't
> a compelling example, I don't think it alone invalidates the idea of
> producing a compile_commands.json file.
>
>
>>
>> - Drive your own AST level source introspection or transformation tools
>>> implemented with clang's LibTooling
>>> <http://clang.llvm.org/docs/LibTooling.html>
>>> - Simplify configuration of emacs flycheck or VI you-complete-me tools
>>> for real-time in-editor syntax checking
>>>
>>> I'm sure there are many more I'm not aware of.
>>>
>>> I acknowledge that this is all fairly clang specific, which is why my
>>> hope had been that it would be possible to generate a compile_commands.json
>>> file in the build system (by writing a SCons Tool), rather than by the
>>> build system (needing to implement it as an intrinsic SCons feature, as I
>>> believe CMake and Ninja do).
>>>
>>> Also, I think it is very important that it be possible to produce the
>>> compilation database without needing to actually compile the entire source
>>> tree. For some projects a complete rebuild may be prohibitively expensive,
>>> but this should not preclude using the above tools. My second attempt above
>>> does work, but has the unwanted side effect that all objects are built, so
>>> does not meet this goal.
>>>
>>
>> I'm pretty sure this can't work in the general case.  It's easy with
>> SCons to create generated headers and sources, for instance, and even
>> introspect sources (which may be generated) to see what needs to be
>> compiled, and how.  (My own day-job build does exactly this, by building a
>> precompiler (written in C++), precompiling a proprietary high-level
>> language into C++ code which is then compiled.) You can't just do an
>> old-school "make -n" and have it be likely to resemble the real list of
>> commands, except in very simple, straightforward cases.
>>
>
> Whether or not it works in the general case is a separate issue. If it
> doesn't work for you, don't use it. The clang tooling ecosystem is widely
> used in practice, and those tools are driven by compilation databases
> produced by build systems like CMake and Ninja. Obviously, there are build
> environments where it is not a viable technique. But surely that isn't an
> argument against making it work with SCons in the cases where it can and
> enables access to a set of very powerful new development tools?
>
>
>>
>> In a simple C++-only single-pass build, I can see this working to some
>> extent.  I suspect the simplest method is to wrap your compiler with a
>> shell script that emits $* to the database before (or after?) running the
>> comand, and use that shell script as $CC/$CXX etc.  Modifying the existing
>> tools as you've started to do could also work, as could overriding the
>> CC*OMSTR variables perhaps.  However, these will only produce the partial
>> results of what actually got compiled -- if you change flags or compiler
>> versions or paths, you could end up with an out-of-sync database.
>>
>
> Right, and this is why I want to do it as something that SCons does
> understand. The shell script approach is not viable for day to day
> development, and as you point out is unlikely to give complete results and
> has synchronization and correctness issues.
>
> Having SCons produce the file as a consequence of walking the graph
> ensures that the genereated compile_commands.json file does reflect reality
> and won't be out of date.
>
> Anyway, thanks for your thoughts on the matter.
>
> Andrew
>
>
>
>>
>> --
>> Gary
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20150111/ff1a5340/attachment.html>