[Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
Daniel Moody
dmoody256 at gmail.com
Mon Apr 18 03:36:03 EDT 2022
Hello liruncong,
I have a test branch for the issues you described where re-running scons is
not re-compiling, could you please try it out?:
https://github.com/dmoody256/scons/tree/ninja_integration_branch
On Sun, Apr 17, 2022 at 2:29 AM liruncong2018 <liruncong2018 at qq.com> wrote:
> If execute the command "scons -j%NUMBER_OF_PROCESSORS% generate-ninja
> --verbose --experimental=ninja", it will be completely recompiled.
> If execute the command again, then it will be completely recompiled.
> Each time is equivalent to the first compilation.
>
> ------------------ 原始邮件 ------------------
> *发件人:* "Daniel Moody" <dmoody256 at gmail.com>;
> *发送时间:* 2022年4月17日(星期天) 中午1:52
> *收件人:* "liruncong2018"<liruncong2018 at qq.com>;
> *抄送:* "Bill Deegan"<bill at baddogconsulting.com>;"SCons users mailing list"<
> scons-users at scons.org>;
> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails
> after using include path file instead of "-I" option
>
> If you use the command line target 'generate-ninja' for the scons command,
> does it still fail to build and execute ninja on the second run?
>
>
> On Sat, Apr 16, 2022 at 10:36 PM liruncong2018 <liruncong2018 at qq.com>
> wrote:
>
>> Great, looking forward to your PR!
>>
>>
>> ------------------ 原始邮件 ------------------
>> *发件人:* "Daniel Moody" <dmoody256 at gmail.com>;
>> *发送时间:* 2022年4月17日(星期天) 中午11:31
>> *收件人:* "liruncong2018"<liruncong2018 at qq.com>;
>> *抄送:* "Bill Deegan"<bill at baddogconsulting.com>;"SCons users mailing
>> list"<scons-users at scons.org>;
>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails
>> after using include path file instead of "-I" option
>>
>> Okay that indicates that scons is not deciding to build and execute the
>> ninja file, probably because nothing has changed in the build scripts when
>> you only change the source file. This is a bug, even if the ninja file is
>> not regened, scons should execute it for you. I will make a PR for this
>> issue. For now you will need to use the run_ninja_env.bat.
>>
>>
>> On Sat, Apr 16, 2022 at 9:54 PM liruncong2018 <liruncong2018 at qq.com>
>> wrote:
>>
>>> Hi,
>>> When I just execute run_ninja_env.bat, it shows compile error. It seems
>>> that there is no problem with ninja, the problem should be that scons does
>>> not call run_ninja_env.bat to execute. Now how do I diagnose this problem
>>> with scons?
>>> ------------------ 原始邮件 ------------------
>>> *发件人:* "Daniel Moody" <dmoody256 at gmail.com>;
>>> *发送时间:* 2022年4月17日(星期天) 上午10:28
>>> *收件人:* "liruncong2018"<liruncong2018 at qq.com>;
>>> *抄送:* "Bill Deegan"<bill at baddogconsulting.com>;"SCons users mailing
>>> list"<scons-users at scons.org>;
>>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails
>>> after using include path file instead of "-I" option
>>>
>>> When I test with the repo https://github.com/liruncong/NinJaTest for
>>> bsp/beaglebone build using the changes from here:
>>> https://github.com/SCons/scons/pull/4133, and I put an 'a' in the
>>> src/irq.c file, I see the file error when I rerun the build.
>>>
>>> Could you specify what file you changed?
>>>
>>> Also on subsequent runs, you can just run_ninja_env.bat, it will skip
>>> scons regeneration, but if you want scons to regen it should work also.
>>>
>>> On Sat, Apr 16, 2022 at 8:46 PM liruncong2018 <liruncong2018 at qq.com>
>>> wrote:
>>>
>>>> Hi,
>>>> After the first compilation is successful, modify the file, expect the
>>>> compilation to fail, but compile again without error message.
>>>> The following is the log after the command("scons
>>>> -j%NUMBER_OF_PROCESSORS% --verbose --experimental=ninja") is executed:
>>>> ______________________________________
>>>> scons: Reading SConscript files ...
>>>> scons: done reading SConscript files.
>>>> scons: Building targets ...
>>>> scons: done building targets.
>>>> ______________________________________
>>>> I have attached the compressed "build.ninja" file in the attachment.
>>>>
>>>> ------------------ 原始邮件 ------------------
>>>> *发件人:* "Daniel Moody" <dmoody256 at gmail.com>;
>>>> *发送时间:* 2022年4月17日(星期天) 凌晨1:53
>>>> *收件人:* "Bill Deegan"<bill at baddogconsulting.com>;
>>>> *抄送:* "SCons users mailing list"<scons-users at scons.org
>>>> >;"liruncong2018"<liruncong2018 at qq.com>;
>>>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation
>>>> fails after using include path file instead of "-I" option
>>>>
>>>> Also i realized that link_objects.txt is probably not meant for the
>>>> object builders. I assume this file is intended for linking the libraries
>>>> and programs? In that case you will want a scanner for those builders
>>>> instead. Do you have error messages you can show, it should help to see
>>>> what your issue is.
>>>>
>>>> On Sat, Apr 16, 2022, 10:58 AM Bill Deegan <bill at baddogconsulting.com>
>>>> wrote:
>>>>
>>>>> try bzip'ing the ninja.build file. That should make it considerably
>>>>> smaller.
>>>>>
>>>>> On Sat, Apr 16, 2022 at 4:37 PM liruncong2018 via Scons-users <
>>>>> scons-users at scons.org> wrote:
>>>>>
>>>>>> Hi,
>>>>>> Still the same problem. I want to send "build.ninja" as an
>>>>>> attachment, but the email always bounces, what information can I print so I
>>>>>> can find the problem?
>>>>>>
>>>>>> ------------------ 原始邮件 ------------------
>>>>>> *发件人:* "Daniel Moody" <dmoody256 at gmail.com>;
>>>>>> *发送时间:* 2022年4月16日(星期六) 凌晨0:29
>>>>>> *收件人:* "liruncong2018"<liruncong2018 at qq.com>;
>>>>>> *抄送:* "SCons users mailing list"<scons-users at scons.org>;
>>>>>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： Ninja compilation fails
>>>>>> after using include path file instead of "-I" option
>>>>>>
>>>>>> Here's the same code but better color scheme:
>>>>>>
>>>>>> incPathsFile = env.Textfile(
>>>>>> "build/__cpp_path.txt",
>>>>>> cppPathStr
>>>>>> )
>>>>>>
>>>>>> defineOptsFile = env.Textfile(
>>>>>> "build/__define_options.txt",
>>>>>> "\n".join(rtconfig.rtconfigObj.DefineOptions)
>>>>>> )
>>>>>>
>>>>>> linkObjsFile = env.Textfile(
>>>>>> "build/__link_objects.txt",
>>>>>> "\n".join(objsList)
>>>>>> )
>>>>>>
>>>>>> def _add_scanner(builder):
>>>>>> def new_scanner(node, env, path):
>>>>>> return incPathsFile + defineOptsFile + linkObjsFile
>>>>>>
>>>>>> builder.builder.target_scanner = SCons.Scanner.Scanner(
>>>>>> function=new_scanner,
>>>>>> path_function=[],
>>>>>> )
>>>>>> for object_builder in SCons.Tool.createObjBuilders(env):
>>>>>> _add_scanner(object_builder)
>>>>>>
>>>>>> On Fri, Apr 15, 2022 at 10:50 AM Daniel Moody <dmoody256 at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> hello liruncong,
>>>>>>>
>>>>>>> If you want to add multiple @file dependencies via the scanner do
>>>>>>> them all in the same scanner. Like I said you can only apply one scanner,
>>>>>>> so in this case you are overwriting each previous scanner. The scanner
>>>>>>> should return a list of files in which to set dependencies for the current
>>>>>>> node.
>>>>>>>
>>>>>>> The path function is not used in this case, and I just left the
>>>>>>> CPPPATH from the default SourceFileScanner I copied the scanner code from.
>>>>>>> It does have to be something besides None I believe though, or scons won't
>>>>>>> scan. The correct way this should be used in this case is you should place
>>>>>>> the @file in a known list of directories or just a single directory, and
>>>>>>> then pass a list of directories or a list containing the single directory,
>>>>>>> and the scanner should find the files by extension in the list of
>>>>>>> directories. This is the fundamental working of scanners if you want to
>>>>>>> read more:
>>>>>>> https://scons.org/doc/production/HTML/scons-user/ch20.html, however
>>>>>>> you could also use a dummy list if there is no dynamic component here.
>>>>>>>
>>>>>>> Here is an updated example based on the new info you provided:
>>>>>>>
>>>>>>> incPathsFile = env.Textfile(
>>>>>>> "build/__cpp_path.txt",
>>>>>>> cppPathStr
>>>>>>> )
>>>>>>>
>>>>>>> defineOptsFile = env.Textfile(
>>>>>>> "build/__define_options.txt",
>>>>>>> "\n".join(rtconfig.rtconfigObj.DefineOptions)
>>>>>>> )
>>>>>>>
>>>>>>> linkObjsFile = env.Textfile(
>>>>>>> "build/__link_objects.txt",
>>>>>>> "\n".join(objsList)
>>>>>>> )
>>>>>>>
>>>>>>> def _add_scanner(builder):
>>>>>>> def new_scanner(node, env, path):
>>>>>>> return incPathsFile + defineOptsFile + linkObjsFile
>>>>>>>
>>>>>>> builder.builder.target_scanner = SCons.Scanner.Scanner(
>>>>>>> function=new_scanner,
>>>>>>> path_function=[],
>>>>>>> )
>>>>>>> for object_builder in SCons.Tool.createObjBuilders(env):
>>>>>>> _add_scanner(object_builder)
>>>>>>>
>>>>>>> On Fri, Apr 15, 2022 at 9:58 AM liruncong2018 <liruncong2018 at qq.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Can you give me more information? What source file? Does the
>>>>>>>> compilation fail the first time? Does it recompile successfully with only
>>>>>>>> scons building? Can you push your latest code so I can see it in your repo?
>>>>>>>> -- I don't know how to reproduce this problem in the git test
>>>>>>>> project, I will try to see if it can be reproduced.
>>>>>>>> Also each builder can only hold one target or source scanner, but
>>>>>>>> you can chain the scanner by calling other scanners from within the single
>>>>>>>> scanner. Are you assigning multiple scanners to the same builder?
>>>>>>>> -- The following are some scanners, I don't quite understand the
>>>>>>>> path_function, especially the second parameter of the last scanner
>>>>>>>> (linkObjsFile).
>>>>>>>> ————————————————————————————————————————
>>>>>>>> incPathsFile = env.Textfile(
>>>>>>>> "build/__cpp_path.txt",
>>>>>>>> cppPathStr
>>>>>>>> )
>>>>>>>> def _add_scanner(builder):
>>>>>>>> def new_scanner(node, env, path):
>>>>>>>> return incPathsFile
>>>>>>>>
>>>>>>>> builder.builder.target_scanner = SCons.Scanner.Scanner(
>>>>>>>> function=new_scanner,
>>>>>>>> path_function=SCons.Script.FindPathDirs('CPPPATH'),
>>>>>>>> )
>>>>>>>> for object_builder in SCons.Tool.createObjBuilders(env):
>>>>>>>> _add_scanner(object_builder)
>>>>>>>>
>>>>>>>> defineOptsFile = env.Textfile(
>>>>>>>> "build/__define_options.txt",
>>>>>>>> "\n".join(rtconfig.rtconfigObj.DefineOptions)
>>>>>>>> )
>>>>>>>>
>>>>>>>> def _add_scanner(builder):
>>>>>>>> def new_scanner(node, env, path):
>>>>>>>> return defineOptsFile
>>>>>>>>
>>>>>>>> builder.builder.target_scanner = SCons.Scanner.Scanner(
>>>>>>>> function=new_scanner,
>>>>>>>> path_function=SCons.Script.FindPathDirs('CPPPATH'),
>>>>>>>> )
>>>>>>>> for object_builder in SCons.Tool.createObjBuilders(env):
>>>>>>>> _add_scanner(object_builder)
>>>>>>>>
>>>>>>>> linkObjsFile = env.Textfile(
>>>>>>>> "build/__link_objects.txt",
>>>>>>>> "\n".join(objsList)
>>>>>>>> )
>>>>>>>>
>>>>>>>> def _add_scanner(builder):
>>>>>>>> def new_scanner(node, env, path):
>>>>>>>> return linkObjsFile
>>>>>>>>
>>>>>>>> builder.builder.target_scanner = SCons.Scanner.Scanner(
>>>>>>>> function=new_scanner,
>>>>>>>> path_function=SCons.Script.FindPathDirs('LIBPATH'),
>>>>>>>> )
>>>>>>>> for object_builder in SCons.Tool.createObjBuilders(env):
>>>>>>>> _add_scanner(object_builder)
>>>>>>>> ————————————————————————————————————————————————
>>>>>>>> ------------------ 原始邮件 ------------------
>>>>>>>> *发件人:* "Daniel Moody" <dmoody256 at gmail.com>;
>>>>>>>> *发送时间:* 2022年4月14日(星期四) 晚上11:36
>>>>>>>> *收件人:* "liruncong2018"<liruncong2018 at qq.com>;
>>>>>>>> *抄送:* "SCons users mailing list"<scons-users at scons.org>;
>>>>>>>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： Ninja compilation
>>>>>>>> fails after using include path file instead of "-I" option
>>>>>>>>
>>>>>>>> Can you give me more information? What source file? Does the
>>>>>>>> compilation fail the first time? Does it recompile successfully with only
>>>>>>>> scons building? Can you push your latest code so I can see it in your repo?
>>>>>>>>
>>>>>>>> Also each builder can only hold one target or source scanner, but
>>>>>>>> you can chain the scanner by calling other scanners from within the single
>>>>>>>> scanner. Are you assigning multiple scanners to the same builder?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Apr 14, 2022 at 10:17 AM liruncong2018 <
>>>>>>>> liruncong2018 at qq.com> wrote:
>>>>>>>>
>>>>>>>>> Hello Daniel Moody,
>>>>>>>>> Thanks a lot for the example you gave. According to your example,
>>>>>>>>> I added 4 scanners, which can indeed be compiled and linked.
>>>>>>>>> But when I edit the source file, I deliberately put "a" at the end
>>>>>>>>> of the .cpp file and expect the compilation to fail. But when I compile
>>>>>>>>> again, nothing compiles.
>>>>>>>>> How to diagnose this problem?
>>>>>>>>>
>>>>>>>>> ------------------ 原始邮件 ------------------
>>>>>>>>> *发件人:* "Daniel Moody" <dmoody256 at gmail.com>;
>>>>>>>>> *发送时间:* 2022年4月14日(星期四) 凌晨2:27
>>>>>>>>> *收件人:* "liruncong2018"<liruncong2018 at qq.com>;
>>>>>>>>> *抄送:* "SCons users mailing list"<scons-users at scons.org>;
>>>>>>>>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： Ninja compilation
>>>>>>>>> fails after using include path file instead of "-I" option
>>>>>>>>>
>>>>>>>>> I added an example how to hook up the dependencies:
>>>>>>>>>
>>>>>>>>> https://github.com/liruncong/NinJaTest/pull/1/files
>>>>>>>>>
>>>>>>>>> On Wed, Apr 13, 2022 at 10:01 AM liruncong2018 <
>>>>>>>>> liruncong2018 at qq.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>> I generate some option files via env.Textfile
>>>>>>>>>> ("build/__cpp_path.txt", "build/__define_options.txt",
>>>>>>>>>> "build/__link_objects.txt", "build/__exe_cpp_path.txt") and make objs
>>>>>>>>>> depend on corresponding files ,for example:
>>>>>>>>>> env.Depends(objs, ["build/__cpp_path.txt",
>>>>>>>>>> "build/__define_options.txt" ])
>>>>>>>>>> env.Depends(target, "build/__link_objects.txt")
>>>>>>>>>> But after enabling ninja, these manually added dependencies are
>>>>>>>>>> ignored, which causes ninja to fail to compile.
>>>>>>>>>> How to deal with this?
>>>>>>>>>>
>>>>>>>>>> ------------------ 原始邮件 ------------------
>>>>>>>>>> *发件人:* "Daniel Moody" <dmoody256 at gmail.com>;
>>>>>>>>>> *发送时间:* 2022年4月13日(星期三) 凌晨0:02
>>>>>>>>>> *收件人:* "SCons users mailing list"<scons-users at scons.org>;
>>>>>>>>>> *抄送:* "liruncong2018"<liruncong2018 at qq.com>;
>>>>>>>>>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： Ninja compilation
>>>>>>>>>> fails after using include path file instead of "-I" option
>>>>>>>>>>
>>>>>>>>>> Hello, that PR is meant to support shared @file with ninja, not
>>>>>>>>>> intended for any improvements to TEMPFILE. Please test without TEMPFILE.
>>>>>>>>>>
>>>>>>>>>> On Tue, Apr 12, 2022 at 10:52 AM liruncong2018 via Scons-users <
>>>>>>>>>> scons-users at scons.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi
>>>>>>>>>>> I tested "https://github.com/SCons/scons/pull/4133", after
>>>>>>>>>>> enabling ningja, except for the link failure, .c/.cpp/.S are successfully
>>>>>>>>>>> compiled into .o, the total time is 6:48 .
>>>>>>>>>>> This time is much greater than the previous time of 3:45 when
>>>>>>>>>>> ninja was not enabled and the option file was shared using @file.
>>>>>>>>>>> After using TEMPFILE instead of @file, the compilation time is
>>>>>>>>>>> basically doubled whether ninja is enabled or not.
>>>>>>>>>>> The TEMPFILE function should be improved to use shared files to
>>>>>>>>>>> prevent the creation of a large number of temporary files.
>>>>>>>>>>>
>>>>>>>>>>> ------------------ 原始邮件 ------------------
>>>>>>>>>>> *发件人:* "SCons users mailing list" <dmoody256 at gmail.com>;
>>>>>>>>>>> *发送时间:* 2022年4月11日(星期一) 晚上11:43
>>>>>>>>>>> *收件人:* "SCons users mailing list"<scons-users at scons.org>;
>>>>>>>>>>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： Ninja compilation fails
>>>>>>>>>>> after using include path file instead of "-I" option
>>>>>>>>>>>
>>>>>>>>>>> I have a potential fix to the ninja tool so that it can decide
>>>>>>>>>>> to use response files or not. This should allow you to use your own shared
>>>>>>>>>>> response file.
>>>>>>>>>>>
>>>>>>>>>>> https://github.com/SCons/scons/pull/4133
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Apr 11, 2022 at 10:01 AM Mats Wichmann <mats at wichmann.us>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 4/11/22 08:57, liruncong2018 via Scons-users wrote:
>>>>>>>>>>>> > Hi,
>>>>>>>>>>>> > Even using msys2 does not solve all problems. The 32K limit
>>>>>>>>>>>> solves my
>>>>>>>>>>>> > compilation problems (.cpp -> .o) but not linking (.o ->
>>>>>>>>>>>> target) because
>>>>>>>>>>>> > the link command line has around 137K (about 2500 .o files).
>>>>>>>>>>>> So
>>>>>>>>>>>> > compilers such as gcc/clang/armclang will support "@file".
>>>>>>>>>>>> > Currently, scons needs to set up Depends when using @file.
>>>>>>>>>>>> This
>>>>>>>>>>>> > dependency suggestion is still directly supported by scons,
>>>>>>>>>>>> and users do
>>>>>>>>>>>> > not need to write dependencies. It should also be supported
>>>>>>>>>>>> when ninja
>>>>>>>>>>>> > is enabled.
>>>>>>>>>>>> > When LINKCOM uses "TEMPFILE", I get a link failure because
>>>>>>>>>>>> armlink.exe
>>>>>>>>>>>> > checks if --cpu is specified on the command line, so it is not
>>>>>>>>>>>> > reasonable for "TEMPFILE" to put all options to a temporary
>>>>>>>>>>>> file.
>>>>>>>>>>>>
>>>>>>>>>>>> Right... thus my suggestion - well, let's call it "musing"
>>>>>>>>>>>> rather than
>>>>>>>>>>>> "suggestion" because this might not work at all:
>>>>>>>>>>>>
>>>>>>>>>>>> > For ninja it currently looks like this:
>>>>>>>>>>>> >
>>>>>>>>>>>> > env["LINKCOM"] = '${TEMPFILE("$LINK $LINKFLAGS
>>>>>>>>>>>> /OUT:$TARGET.windows
>>>>>>>>>>>> > $_LIBDIRFLAGS $_LIBFLAGS $_PDB $SOURCES.windows",
>>>>>>>>>>>> "$LINKCOMSTR")}'
>>>>>>>>>>>> >
>>>>>>>>>>>> > env["SHLINKCOM"] = '${TEMPFILE("$SHLINK $SHLINKFLAGS
>>>>>>>>>>>> $_SHLINK_TARGETS
>>>>>>>>>>>> > $_LIBDIRFLAGS $_LIBFLAGS $_PDB $_SHLINK_SOURCES",
>>>>>>>>>>>> "$SHLINKCOMSTR")}'
>>>>>>>>>>>> >
>>>>>>>>>>>> > So I'm curious if moving $LINK $LINKFLAGS to the left of
>>>>>>>>>>>> ${TEMPFILE
>>>>>>>>>>>> > would help...
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Scons-users mailing list
>>>>>>>>>>>> Scons-users at scons.org
>>>>>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Scons-users mailing list
>>>>>>>>>>> Scons-users at scons.org
>>>>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>> Scons-users mailing list
>>>>>> Scons-users at scons.org
>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>
>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20220418/442c4b79/attachment-0001.htm>
More information about the Scons-users
mailing list