[Scons-users] TempFileMunge conflicts with ccache

[Daniel Moody]

I was recently messing with ccache and ran into this, and I noticed a
github issue was never created (or I couldn't find one:
https://github.com/SCons/scons/issues?q=is%3Aissue+ccache)

So this happens if you set the "compiler" (e.g. $CXX) to a two word
statement like for example:

env['CXX'] = 'ccache ' +  env['CXX']

The problem with this approach is that this gets wrapped into the tempfile
command itself, for example in the msvc tool:
https://github.com/SCons/scons/blob/5a438e635699dfee672f0bcbbef79b57e79b4bcd/SCons/Tool/msvc.py#L254


Seems the tempfile code is expecting the "compiler" to be a single word
argument from this line where it takes only the first arg based off spaces
in the cmd (it seems to assume that cmd[0] is equal to $CXX in the above
example):
https://github.com/SCons/scons/blob/5a438e635699dfee672f0bcbbef79b57e79b4bcd/SCons/Platform/__init__.py#L252


There is a question here if setting a "compiler" to a two word statement is
allowable? I didn't find any documentation acknowledging it or forbidding
it, but it seems to happen to work in some cases.

The above example will work if you remove ccache out of the "compiler"
statement and put it into the full command statement like this for example:

env['CXXCOM'] = 'ccache ' +  env['CXXCOM']

That results in tempfile link being correctly given to the "compiler" and
'ccache' still sitting out in front of the final command like it is
supposed to. This does have a draw back that you can't set CXX from the
command line potentially, or rather it would be difficult to set CXXCOM
from the command line.

So seems there should be a github issue to update either tempfile to work
with two word "compilers"  OR documentation discouraging or denying support
for two word compiler statements


On Wed, Dec 19, 2018 at 10:45 AM Andrew C. Morrow <andrew.c.morrow at gmail.com>
wrote:

>
> In this case, I'm sort of just relaying word from someone who was trying
> to build MongoDB this way. I don't use ccache, I use the SCons cache. But I
> thought the assumption baked into Action was worth flagging.
>
> On Wed, Dec 19, 2018 at 10:12 AM Mats Wichmann <mats at wichmann.us> wrote:
>
>> On 12/19/18 8:03 AM, Andrew C. Morrow wrote:
>> > While investigating this issue, I think I discovered a potentially more
>> > serious issue. Evaluation of a build where CC is set to a multi token
>> value
>> > like ccache gcc will not rebuild files if the compiler is upgraded,
>> because
>> > CommandAction.get_implicit_deps only considers the first token of the
>> > command line as an implicit dependency which must be checked for
>> > up-to-date-ness. In the case of CC="ccache gcc" build, that would
>> result in
>> > a rebuild when the ccache binary changed, but not when the gcc binary
>> > changed.
>> >
>> > I'm not sure if there is anything really that can be done about that,
>> but
>> > maybe some guard rails around setting tool variables to multi token
>> values
>> > would be a good idea?
>> >
>> >
>> >
>> > On Thu, Dec 6, 2018 at 12:14 PM Andrew C. Morrow <
>> andrew.c.morrow at gmail.com>
>> > wrote:
>> >
>> >>
>> >> In the MongoDB build, we enable TempFileMunge for non-windows
>> platforms,
>> >> since sometimes we have truly astronomically long link lines which
>> break
>> >> even the linux command line length limitations.
>> >>
>> >> https://github.com/mongodb/mongo/blob/master/SConstruct#L1478-L1496
>> >>
>> >> A user reported that their builds failed when trying to use ccache:
>> >> https://jira.mongodb.org/browse/SERVER-38389
>> >>
>> >> It appears that TempFileMunge just assumes that only the first token of
>> >> the command is actually the "compiler", and ends up feeding the
>> response
>> >> file directly to ccache, rather than invoking, say `ccache
>> >> g++ @/path/to/temp/file`.
>> >>
>> >> On the other hand, could this even work in principle? Is ccache smart
>> >> enough to know to "see into" the response file in order to interact
>> with
>> >> its cache correctly? This CMake bug suggests not:
>> >> https://github.com/kripken/emscripten/issues/4750
>> >>
>> >> I did steer the user towards using the SCons cache instead of ccache,
>> and
>> >> provided a workaround of setting MAXLINELENGTH=<huge-number>
>> >>
>> >> At the very least, if ccache and response files can't be mixed, perhaps
>> >> TempFileMunge could include some guard rails here to bail out if it
>> sees
>> >> the first token to be ccache.
>> >>
>> >> Thoughts?
>>
>>
>> Isn't the more normal usage model to let ccache masquerade as gcc, in
>> which case there's only one compiler token value on the line?
>>
>> That is, from the manpage:
>>
>> SYNOPSIS
>>        ...
>>        ccache compiler [compiler options]
>>        compiler [compiler options]                   (via symbolic link)
>>
>> use the second form?  that doesn't necessarily excuse scons not
>> understanding about the other scenario, but could provide a way forward.
>>
>>
>>
>>
>> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20200511/3af5c63f/attachment.html>