[Scons-users] 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option

[Bill Deegan]

How long is your normally expanded compile command line (not using
@__cpp_path.txt) ?
If you're building with a flavor of mingw (in msys shell?), what's that
shells maximum?
You could potentially change the shell being used by SCons and avoid the
need for response files altogether..

-Bill

On Sun, Apr 10, 2022 at 12:07 AM liruncong2018 via Scons-users <
scons-users at scons.org> wrote:
>
> Hello Daniel Moody,
> If using ninja and still not using shared files, then this should also be
optimized. At present, after I changed it to "TEMPFILE", it still failed to
compile, and I can't test whether the efficiency of enabling ninja is
temporarily.
> However, if ninja is enabled, my project still has to create thousands of
temporary files, I think, this has the same efficiency problem as not
enabling ninja.
>
>
> ------------------ 原始邮件 ------------------
> 发件人: "Daniel Moody" <dmoody256 at gmail.com>;
> 发送时间: 2022年4月10日(星期天) 下午2:14
> 收件人: "liruncong2018"<liruncong2018 at qq.com>;
> 抄送: "SCons users mailing list"<scons-users at scons.org>;
> 主题: Re: [Scons-users] 回复： Ninja compilation fails after using include
path file instead of "-I" option
>
> just to clarify:
>
> you will need to use the @__cpp_path.txt file conditionally if ninja is
not in use
>
> On Sun, Apr 10, 2022 at 1:12 AM Daniel Moody <dmoody256 at gmail.com> wrote:
>>
>> I see, I didn't realize you were sharing the file.
>>
>> The ninja tool will always use response files for the C/C++ related
tools, so in this case you will need to use the tempfile conditionally if
ninja is not in use. We have a line item to make response files conditional
for ninja in this issue: https://github.com/SCons/scons/issues/3929 but it
is not yet implemented.
>>
>> Using a shared response file could be an integrated feature for scons,
but TEMPFILE is not meant for that use case. For now, a shared response
file will need to be done manually on the command line like you are doing.
>>
>> On Sat, Apr 9, 2022 at 10:13 PM liruncong2018 <liruncong2018 at qq.com>
wrote:
>>>
>>> Hello Daniel Moody,
>>> The difference is that when "@file" is used, a temporary file is shared
at compile time. When using "TEMPFILE", each compile command creates a
temporary file, which in my case resulted in thousands of temporary files.
>>> After canceling the MAXLINELENGTH setting, there is no time change, it
is still 3 minutes longer than using "@file".
>>>
>>>
>>> ------------------ 原始邮件 ------------------
>>> 发件人: "liruncong2018" <liruncong2018 at qq.com>;
>>> 发送时间: 2022年4月10日(星期天) 中午11:11
>>> 收件人: "Daniel Moody"<dmoody256 at gmail.com>;
>>> 抄送: "SCons users mailing list"<scons-users at scons.org>;
>>> 主题: 回复： [Scons-users] 回复： Ninja compilation fails after using include
path file instead of "-I" option
>>>
>>> Hello Daniel Moody,
>>> The difference is that when "@file" is used, a temporary file is shared
at compile time. When using "TEMPFILE", each compile command creates a
temporary file, which in my case resulted in thousands of temporary files.
>>>
>>>
>>> ------------------ 原始邮件 ------------------
>>> 发件人: "Daniel Moody" <dmoody256 at gmail.com>;
>>> 发送时间: 2022年4月10日(星期天) 上午10:56
>>> 收件人: "SCons users mailing list"<scons-users at scons.org>;
>>> 抄送: "liruncong2018"<liruncong2018 at qq.com>;
>>> 主题: Re: [Scons-users] 回复： Ninja compilation fails after using include
path file instead of "-I" option
>>>
>>> liruncong,
>>>
>>> Using @file also means temporary files are in use, so there is no
difference in terms of temporary files.
>>>
>>> In my diff, I included several statements meant for debugging and
example demonstration, for example setting the maxlinelength. SCons will
detect this automatically and you should not set it under normal
circumstances. Also I set the tempfile to be verbose, but it is not needed.
SCons will only use temporary files if it is needed, so there's not really
another option, if the temporary file is not used, the compilation will
fail. Please make sure you are not setting the maxlinelength.
>>>
>>>
>>> On Sat, Apr 9, 2022 at 8:30 PM liruncong2018 via Scons-users <
scons-users at scons.org> wrote:
>>>>
>>>> The "TEMPFILE" function should be improved to improve compilation
efficiency. At present, we have tested our actual project and changed
"@file" to "TEMPFILE". When ninja is not enabled, the compilation time
(j32) is increased from 3:45 to 6:30. The duration has increased by almost
3 minutes, which should be related to the massive creation of temporary
files. If put "-o $TARGET" and "$SOURCE" directly on the command line
without writing temporary files, the number of temporary files generated in
large numbers can be as few as 2~3.
>>>>
>>>>
>>>> ------------------ 原始邮件 ------------------
>>>> 发件人: "SCons users mailing list" <bill at baddogconsulting.com>;
>>>> 发送时间: 2022年4月10日(星期天) 凌晨2:31
>>>> 收件人: "SCons users mailing list"<scons-users at scons.org>;
>>>> 主题: Re: [Scons-users] Ninja compilation fails after using include path
file instead of "-I" option
>>>>
>>>> Daniel,
>>>>
>>>> Is it likely we should define TEMPFILE for mingw as we do for win32?
>>>>
>>>> -Bill
>>>>
>>>> On Sat, Apr 9, 2022 at 12:39 AM Daniel Moody <dmoody256 at gmail.com>
wrote:
>>>>>
>>>>> hello liruncong,
>>>>>
>>>>> You can not directly use the @ response file syntax with ninja, you
should instead use SCons internal feature for this, TEMPFILE, as the ninja
tool will know how to process that.
>>>>>
>>>>> TEMPFILE is meant to be substituted in the full command line, here is
an example how to make your CC compile commands use the tempfile:
>>>>>
>>>>> env['_RT_CCCOM'] = env['CCCOM']
>>>>> env['CCCOM'] = "${TEMPFILE('$_RT_CCCOM', '$CCCOMSTR')}"
>>>>>
>>>>> Here is a diff of your project where I am using the response file in
scons and it works with ninja:
>>>>>
>>>>> diff --git a/bsp/beaglebone/SConstruct b/bsp/beaglebone/SConstruct
>>>>> index e9eceb553..01c27a83f 100644
>>>>> --- a/bsp/beaglebone/SConstruct
>>>>> +++ b/bsp/beaglebone/SConstruct
>>>>> @@ -56,8 +56,26 @@ def GenIncPathsFile(env, objs):
>>>>>      env.Depends(objs, [incPathsFile])
>>>>>      return incPathsFile
>>>>>
>>>>> -incPathsFile = GenIncPathsFile(env, objs)
>>>>> -env["_CPPINCFLAGS"] = "@%s" % (incPathsFile)
>>>>> +env['_RT_CCCOM'] = env['CCCOM']
>>>>> +env['CCCOM'] = "${TEMPFILE('$_RT_CCCOM', '$CCCOMSTR')}"
>>>>> +
>>>>> +# This is only to see the output command using the response file if
building with scons
>>>>> +env['CCCOMSTR'] = None
>>>>> +
>>>>> +# My system did not hit the limit in order to use response file,
>>>>> +# so I had to force it to make sure
>>>>> +# I could see the response files in use by scons
>>>>> +env['MAXLINELENGTH'] = 10
>>>>> +
>>>>> +# This function make the response file use the correct slash for
mingw
>>>>> +from SCons.Subst import quote_spaces
>>>>> +def tempfile_arg_esc_func(arg):
>>>>> +    arg = quote_spaces(arg)
>>>>> +    return arg.replace('\\', '/')
>>>>> +env["TEMPFILEARGESCFUNC"] = tempfile_arg_esc_func
>>>>> +
>>>>> +#incPathsFile = GenIncPathsFile(env, objs)
>>>>> +#env["_CPPINCFLAGS"] = "@%s" % (incPathsFile)
>>>>>
>>>>>  # make a building
>>>>>  DoBuilding(TARGET, objs)
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Apr 8, 2022 at 7:23 PM liruncong2018 via Scons-users <
scons-users at scons.org> wrote:
>>>>>>
>>>>>> git repository: https://github.com/liruncong/NinJaTest.git bsp:
beaglebone
>>>>>> mingw:
https://download-sh-cmcc.rt-thread.org:9151/www/aozima/env_released_1%20.2.0.7z
gcc position : env\tools\gnu_gcc\arm_gcc\mingw\bin
>>>>>> python: 3.8.5(anaconda)
>>>>>> platform: win11
>>>>>> scons: https://github.com/SCons/scons.git
>>>>>>
>>>>>> Directly using the "scons" command can compile successfully, but
using "scons --experimental=ninja" will fail to compile. Ninja compilation
fails after using include path file instead of "-I" option · Issue #4130 ·
SCons/scons (github.com)
>>>>>>
>>>>>> Due to the length limit of the windows command line, in our actual
project, the command will exceed the length limit of the windows command
line, so this test case uses the "@file" option.
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Scons-users mailing list
>>>>>> Scons-users at scons.org
>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20220410/2612e865/attachment.htm>