[Scons-users] 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
liruncong2018
liruncong2018 at qq.com
Mon Apr 11 09:18:02 EDT 2022
How long is your normally expanded compile command line (not using @__cpp_path.txt) ? -- About 16K bytes(armclang cmd line) or 137K bytes(armlink cmd line).
If you're building with a flavor of mingw (in msys shell?), what's that shells maximum? -- The actual compiler armclang, executed on windows, windows cmd supports about 2047 bytes.
You could potentially change the shell being used by SCons and avoid the need for response files altogether. -- Wonder if an unrestricted shell exists and can be used on windows? cmd.exe is a widely used shell on windows.
In addition, the "TEMPFILE" method cannot be linked here, because the linker armlink checks whether there is the --cpu option on the command line, and if not, an error is reported. "TEMPFILE" puts all options in a temporary file, so linking fails.
------------------ 原始邮件 ------------------
发件人: "Bill Deegan" <bill at baddogconsulting.com>;
发送时间: 2022年4月11日(星期一) 凌晨4:53
收件人: "SCons users mailing list"<scons-users at scons.org>;
抄送: "Daniel Moody"<dmoody256 at gmail.com>;"liruncong2018"<liruncong2018 at qq.com>;
主题: Re: [Scons-users] 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
How long is your normally expanded compile command line (not using @__cpp_path.txt) ?
If you're building with a flavor of mingw (in msys shell?), what's that shells maximum?
You could potentially change the shell being used by SCons and avoid the need for response files altogether..
-Bill
On Sun, Apr 10, 2022 at 12:07 AM liruncong2018 via Scons-users <scons-users at scons.org> wrote:
>
> Hello Daniel Moody,
> If using ninja and still not using shared files, then this should also be optimized. At present, after I changed it to "TEMPFILE", it still failed to compile, and I can't test whether the efficiency of enabling ninja is temporarily.
> However, if ninja is enabled, my project still has to create thousands of temporary files, I think, this has the same efficiency problem as not enabling ninja.
>
>
> ------------------ 原始邮件 ------------------
> 发件人: "Daniel Moody" <dmoody256 at gmail.com>;
> 发送时间: 2022年4月10日(星期天) 下午2:14
> 收件人: "liruncong2018"<liruncong2018 at qq.com>;
> 抄送: "SCons users mailing list"<scons-users at scons.org>;
> 主题: Re: [Scons-users] 回复： Ninja compilation fails after using include path file instead of "-I" option
>
> just to clarify:
>
> you will need to use the @__cpp_path.txt file conditionally if ninja is not in use
>
> On Sun, Apr 10, 2022 at 1:12 AM Daniel Moody <dmoody256 at gmail.com> wrote:
>>
>> I see, I didn't realize you were sharing the file.
>>
>> The ninja tool will always use response files for the C/C++ related tools, so in this case you will need to use the tempfile conditionally if ninja is not in use. We have a line item to make response files conditional for ninja in this issue: https://github.com/SCons/scons/issues/3929 but it is not yet implemented.
>>
>> Using a shared response file could be an integrated feature for scons, but TEMPFILE is not meant for that use case. For now, a shared response file will need to be done manually on the command line like you are doing.
>>
>> On Sat, Apr 9, 2022 at 10:13 PM liruncong2018 <liruncong2018 at qq.com> wrote:
>>>
>>> Hello Daniel Moody,
>>> The difference is that when "@file" is used, a temporary file is shared at compile time. When using "TEMPFILE", each compile command creates a temporary file, which in my case resulted in thousands of temporary files.
>>> After canceling the MAXLINELENGTH setting, there is no time change, it is still 3 minutes longer than using "@file".
>>>
>>>
>>> ------------------ 原始邮件 ------------------
>>> 发件人: "liruncong2018" <liruncong2018 at qq.com>;
>>> 发送时间: 2022年4月10日(星期天) 中午11:11
>>> 收件人: "Daniel Moody"<dmoody256 at gmail.com>;
>>> 抄送: "SCons users mailing list"<scons-users at scons.org>;
>>> 主题: 回复： [Scons-users] 回复： Ninja compilation fails after using include path file instead of "-I" option
>>>
>>> Hello Daniel Moody,
>>> The difference is that when "@file" is used, a temporary file is shared at compile time. When using "TEMPFILE", each compile command creates a temporary file, which in my case resulted in thousands of temporary files.
>>>
>>>
>>> ------------------ 原始邮件 ------------------
>>> 发件人: "Daniel Moody" <dmoody256 at gmail.com>;
>>> 发送时间: 2022年4月10日(星期天) 上午10:56
>>> 收件人: "SCons users mailing list"<scons-users at scons.org>;
>>> 抄送: "liruncong2018"<liruncong2018 at qq.com>;
>>> 主题: Re: [Scons-users] 回复： Ninja compilation fails after using include path file instead of "-I" option
>>>
>>> liruncong,
>>>
>>> Using @file also means temporary files are in use, so there is no difference in terms of temporary files.
>>>
>>> In my diff, I included several statements meant for debugging and example demonstration, for example setting the maxlinelength. SCons will detect this automatically and you should not set it under normal circumstances. Also I set the tempfile to be verbose, but it is not needed. SCons will only use temporary files if it is needed, so there's not really another option, if the temporary file is not used, the compilation will fail. Please make sure you are not setting the maxlinelength.
>>>
>>>
>>> On Sat, Apr 9, 2022 at 8:30 PM liruncong2018 via Scons-users <scons-users at scons.org> wrote:
>>>>
>>>> The "TEMPFILE" function should be improved to improve compilation efficiency. At present, we have tested our actual project and changed "@file" to "TEMPFILE". When ninja is not enabled, the compilation time (j32) is increased from 3:45 to 6:30. The duration has increased by almost 3 minutes, which should be related to the massive creation of temporary files. If put "-o $TARGET" and "$SOURCE" directly on the command line without writing temporary files, the number of temporary files generated in large numbers can be as few as 2~3.
>>>>
>>>>
>>>> ------------------ 原始邮件 ------------------
>>>> 发件人: "SCons users mailing list" <bill at baddogconsulting.com>;
>>>> 发送时间: 2022年4月10日(星期天) 凌晨2:31
>>>> 收件人: "SCons users mailing list"<scons-users at scons.org>;
>>>> 主题: Re: [Scons-users] Ninja compilation fails after using include path file instead of "-I" option
>>>>
>>>> Daniel,
>>>>
>>>> Is it likely we should define TEMPFILE for mingw as we do for win32?
>>>>
>>>> -Bill
>>>>
>>>> On Sat, Apr 9, 2022 at 12:39 AM Daniel Moody <dmoody256 at gmail.com> wrote:
>>>>>
>>>>> hello liruncong,
>>>>>
>>>>> You can not directly use the @ response file syntax with ninja, you should instead use SCons internal feature for this, TEMPFILE, as the ninja tool will know how to process that.
>>>>>
>>>>> TEMPFILE is meant to be substituted in the full command line, here is an example how to make your CC compile commands use the tempfile:
>>>>>
>>>>> env['_RT_CCCOM'] = env['CCCOM']
>>>>> env['CCCOM'] = "${TEMPFILE('$_RT_CCCOM', '$CCCOMSTR')}"
>>>>>
>>>>> Here is a diff of your project where I am using the response file in scons and it works with ninja:
>>>>>
>>>>> diff --git a/bsp/beaglebone/SConstruct b/bsp/beaglebone/SConstruct
>>>>> index e9eceb553..01c27a83f 100644
>>>>> --- a/bsp/beaglebone/SConstruct
>>>>> +++ b/bsp/beaglebone/SConstruct
>>>>> @@ -56,8 +56,26 @@ def GenIncPathsFile(env, objs):
>>>>> env.Depends(objs, [incPathsFile])
>>>>> return incPathsFile
>>>>>
>>>>> -incPathsFile = GenIncPathsFile(env, objs)
>>>>> -env["_CPPINCFLAGS"] = "@%s" % (incPathsFile)
>>>>> +env['_RT_CCCOM'] = env['CCCOM']
>>>>> +env['CCCOM'] = "${TEMPFILE('$_RT_CCCOM', '$CCCOMSTR')}"
>>>>> +
>>>>> +# This is only to see the output command using the response file if building with scons
>>>>> +env['CCCOMSTR'] = None
>>>>> +
>>>>> +# My system did not hit the limit in order to use response file,
>>>>> +# so I had to force it to make sure
>>>>> +# I could see the response files in use by scons
>>>>> +env['MAXLINELENGTH'] = 10
>>>>> +
>>>>> +# This function make the response file use the correct slash for mingw
>>>>> +from SCons.Subst import quote_spaces
>>>>> +def tempfile_arg_esc_func(arg):
>>>>> + arg = quote_spaces(arg)
>>>>> + return arg.replace('\\', '/')
>>>>> +env["TEMPFILEARGESCFUNC"] = tempfile_arg_esc_func
>>>>> +
>>>>> +#incPathsFile = GenIncPathsFile(env, objs)
>>>>> +#env["_CPPINCFLAGS"] = "@%s" % (incPathsFile)
>>>>>
>>>>> # make a building
>>>>> DoBuilding(TARGET, objs)
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Apr 8, 2022 at 7:23 PM liruncong2018 via Scons-users <scons-users at scons.org> wrote:
>>>>>>
>>>>>> git repository: https://github.com/liruncong/NinJaTest.git bsp: beaglebone
>>>>>> mingw: https://download-sh-cmcc.rt-thread.org:9151/www/aozima/env_released_1%20.2.0.7z gcc position : env\tools\gnu_gcc\arm_gcc\mingw\bin
>>>>>> python: 3.8.5(anaconda)
>>>>>> platform: win11
>>>>>> scons: https://github.com/SCons/scons.git
>>>>>>
>>>>>> Directly using the "scons" command can compile successfully, but using "scons --experimental=ninja" will fail to compile. Ninja compilation fails after using include path file instead of "-I" option · Issue #4130 · SCons/scons (github.com)
>>>>>>
>>>>>> Due to the length limit of the windows command line, in our actual project, the command will exceed the length limit of the windows command line, so this test case uses the "@file" option.
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Scons-users mailing list
>>>>>> Scons-users at scons.org
>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20220411/a3a823d9/attachment-0001.htm>
More information about the Scons-users
mailing list