[Scons-users] 回复：回复：回复：回复：回复： Ninja compilation fails after using include path file instead of "-I" option

Fri Apr 15 12:29:28 EDT 2022

Here's the same code but better color scheme:

incPathsFile = env.Textfile(
    "build/__cpp_path.txt",
    cppPathStr
)

defineOptsFile = env.Textfile(
    "build/__define_options.txt",
    "\n".join(rtconfig.rtconfigObj.DefineOptions)
)

linkObjsFile = env.Textfile(
    "build/__link_objects.txt",
    "\n".join(objsList)
)

def _add_scanner(builder):
    def new_scanner(node, env, path):
        return incPathsFile + defineOptsFile + linkObjsFile

    builder.builder.target_scanner = SCons.Scanner.Scanner(
        function=new_scanner,
        path_function=[],
    )
for object_builder in SCons.Tool.createObjBuilders(env):
    _add_scanner(object_builder)

On Fri, Apr 15, 2022 at 10:50 AM Daniel Moody <dmoody256 at gmail.com> wrote:

> hello liruncong,
>
> If you want to add multiple @file dependencies via the scanner do them all
> in the same scanner. Like I said you can only apply one scanner, so in this
> case you are overwriting each previous scanner. The scanner should return a
> list of files in which to set dependencies for the current node.
>
> The path function is not used in this case, and I just left the CPPPATH
> from the default SourceFileScanner I copied the scanner code from. It does
> have to be something besides None I believe though, or scons won't scan.
> The correct way this should be used in this case is you should place
> the @file in a known list of directories or just a single directory, and
> then pass a list of directories or a list containing the single directory,
> and the scanner should find the files by extension in the list of
> directories. This is the fundamental working of scanners if you want to
> read more: https://scons.org/doc/production/HTML/scons-user/ch20.html,
> however you could also use a dummy list if there is no dynamic component
> here.
>
> Here is an updated example based on the new info you provided:
>
> incPathsFile = env.Textfile(
>     "build/__cpp_path.txt",
>     cppPathStr
> )
>
> defineOptsFile = env.Textfile(
>     "build/__define_options.txt",
>     "\n".join(rtconfig.rtconfigObj.DefineOptions)
> )
>
> linkObjsFile = env.Textfile(
>     "build/__link_objects.txt",
>     "\n".join(objsList)
> )
>
> def _add_scanner(builder):
>     def new_scanner(node, env, path):
>         return incPathsFile + defineOptsFile + linkObjsFile
>
>     builder.builder.target_scanner = SCons.Scanner.Scanner(
>         function=new_scanner,
>         path_function=[],
>     )
> for object_builder in SCons.Tool.createObjBuilders(env):
>     _add_scanner(object_builder)
>
> On Fri, Apr 15, 2022 at 9:58 AM liruncong2018 <liruncong2018 at qq.com>
> wrote:
>
>> Can you give me more information? What source file? Does the compilation
>> fail the first time? Does it recompile successfully with only scons
>> building? Can you push your latest code so I can see it in your repo?
>> -- I don't know how to reproduce this problem in the git test project, I
>> will try to see if it can be reproduced.
>> Also each builder can only hold one target or source scanner, but you can
>> chain the scanner by calling other scanners from within the single scanner.
>> Are you assigning multiple scanners to the same builder?
>> -- The following are some scanners, I don't quite understand the
>> path_function, especially the second parameter of the last scanner
>> (linkObjsFile).
>> ————————————————————————————————————————
>>     incPathsFile = env.Textfile(
>>         "build/__cpp_path.txt",
>>         cppPathStr
>>     )
>>     def _add_scanner(builder):
>>         def new_scanner(node, env, path):
>>             return incPathsFile
>>
>>         builder.builder.target_scanner = SCons.Scanner.Scanner(
>>             function=new_scanner,
>>             path_function=SCons.Script.FindPathDirs('CPPPATH'),
>>         )
>>     for object_builder in SCons.Tool.createObjBuilders(env):
>>         _add_scanner(object_builder)
>>
>>     defineOptsFile = env.Textfile(
>>         "build/__define_options.txt",
>>         "\n".join(rtconfig.rtconfigObj.DefineOptions)
>>     )
>>
>>     def _add_scanner(builder):
>>         def new_scanner(node, env, path):
>>             return defineOptsFile
>>
>>         builder.builder.target_scanner = SCons.Scanner.Scanner(
>>             function=new_scanner,
>>             path_function=SCons.Script.FindPathDirs('CPPPATH'),
>>         )
>>     for object_builder in SCons.Tool.createObjBuilders(env):
>>         _add_scanner(object_builder)
>>
>>     linkObjsFile = env.Textfile(
>>         "build/__link_objects.txt",
>>         "\n".join(objsList)
>>     )
>>
>>     def _add_scanner(builder):
>>         def new_scanner(node, env, path):
>>             return linkObjsFile
>>
>>         builder.builder.target_scanner = SCons.Scanner.Scanner(
>>             function=new_scanner,
>>             path_function=SCons.Script.FindPathDirs('LIBPATH'),
>>         )
>>     for object_builder in SCons.Tool.createObjBuilders(env):
>>         _add_scanner(object_builder)
>> ————————————————————————————————————————————————
>> ------------------ 原始邮件 ------------------
>> *发件人:* "Daniel Moody" <dmoody256 at gmail.com>;
>> *发送时间:* 2022年4月14日(星期四) 晚上11:36
>> *收件人:* "liruncong2018"<liruncong2018 at qq.com>;
>> *抄送:* "SCons users mailing list"<scons-users at scons.org>;
>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： Ninja compilation fails
>> after using include path file instead of "-I" option
>>
>> Can you give me more information? What source file? Does the compilation
>> fail the first time? Does it recompile successfully with only scons
>> building? Can you push your latest code so I can see it in your repo?
>>
>>  Also each builder can only hold one target or source scanner, but you
>> can chain the scanner by calling other scanners from within the single
>> scanner. Are you assigning multiple scanners to the same builder?
>>
>>
>>
>> On Thu, Apr 14, 2022 at 10:17 AM liruncong2018 <liruncong2018 at qq.com>
>> wrote:
>>
>>> Hello Daniel Moody,
>>> Thanks a lot for the example you gave. According to your example, I
>>> added 4 scanners, which can indeed be compiled and linked.
>>> But when I edit the source file, I deliberately put "a" at the end of
>>> the .cpp file and expect the compilation to fail. But when I compile again,
>>> nothing compiles.
>>> How to diagnose this problem?
>>>
>>> ------------------ 原始邮件 ------------------
>>> *发件人:* "Daniel Moody" <dmoody256 at gmail.com>;
>>> *发送时间:* 2022年4月14日(星期四) 凌晨2:27
>>> *收件人:* "liruncong2018"<liruncong2018 at qq.com>;
>>> *抄送:* "SCons users mailing list"<scons-users at scons.org>;
>>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： Ninja compilation fails
>>> after using include path file instead of "-I" option
>>>
>>> I added an example how to hook up the dependencies:
>>>
>>> https://github.com/liruncong/NinJaTest/pull/1/files
>>>
>>> On Wed, Apr 13, 2022 at 10:01 AM liruncong2018 <liruncong2018 at qq.com>
>>> wrote:
>>>
>>>> Hi,
>>>> I generate some option files via env.Textfile ("build/__cpp_path.txt",
>>>> "build/__define_options.txt", "build/__link_objects.txt",
>>>> "build/__exe_cpp_path.txt") and make objs depend on corresponding files
>>>> ,for example:
>>>> env.Depends(objs, ["build/__cpp_path.txt", "build/__define_options.txt"
>>>> ])
>>>> env.Depends(target, "build/__link_objects.txt")
>>>> But after enabling ninja, these manually added dependencies are
>>>> ignored, which causes ninja to fail to compile.
>>>> How to deal with this?
>>>>
>>>> ------------------ 原始邮件 ------------------
>>>> *发件人:* "Daniel Moody" <dmoody256 at gmail.com>;
>>>> *发送时间:* 2022年4月13日(星期三) 凌晨0:02
>>>> *收件人:* "SCons users mailing list"<scons-users at scons.org>;
>>>> *抄送:* "liruncong2018"<liruncong2018 at qq.com>;
>>>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： Ninja compilation fails
>>>> after using include path file instead of "-I" option
>>>>
>>>> Hello, that PR is meant to support shared @file with ninja, not
>>>> intended for any improvements to TEMPFILE. Please test without TEMPFILE.
>>>>
>>>> On Tue, Apr 12, 2022 at 10:52 AM liruncong2018 via Scons-users <
>>>> scons-users at scons.org> wrote:
>>>>
>>>>> Hi
>>>>> I tested "https://github.com/SCons/scons/pull/4133", after enabling
>>>>> ningja, except for the link failure, .c/.cpp/.S are successfully compiled
>>>>> into .o, the total time is 6:48 .
>>>>> This time is much greater than the previous time of 3:45 when ninja
>>>>> was not enabled and the option file was shared using @file.
>>>>> After using TEMPFILE instead of @file, the compilation time is
>>>>> basically doubled whether ninja is enabled or not.
>>>>> The TEMPFILE function should be improved to use shared files to
>>>>> prevent the creation of a large number of temporary files.
>>>>>
>>>>> ------------------ 原始邮件 ------------------
>>>>> *发件人:* "SCons users mailing list" <dmoody256 at gmail.com>;
>>>>> *发送时间:* 2022年4月11日(星期一) 晚上11:43
>>>>> *收件人:* "SCons users mailing list"<scons-users at scons.org>;
>>>>> *主题:* Re: [Scons-users] 回复： 回复： 回复： 回复： Ninja compilation fails after
>>>>> using include path file instead of "-I" option
>>>>>
>>>>> I have a potential fix to the ninja tool so that it can decide to use
>>>>> response files or not. This should allow you to use your own shared
>>>>> response file.
>>>>>
>>>>> https://github.com/SCons/scons/pull/4133
>>>>>
>>>>> On Mon, Apr 11, 2022 at 10:01 AM Mats Wichmann <mats at wichmann.us>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>> On 4/11/22 08:57, liruncong2018 via Scons-users wrote:
>>>>>> > Hi,
>>>>>> > Even using msys2 does not solve all problems. The 32K limit solves
>>>>>> my
>>>>>> > compilation problems (.cpp -> .o) but not linking (.o -> target)
>>>>>> because
>>>>>> > the link command line has around 137K (about 2500 .o files). So
>>>>>> > compilers such as gcc/clang/armclang will support "@file".
>>>>>> > Currently, scons needs to set up Depends when using @file. This
>>>>>> > dependency suggestion is still directly supported by scons, and
>>>>>> users do
>>>>>> > not need to write dependencies. It should also be supported when
>>>>>> ninja
>>>>>> > is enabled.
>>>>>> > When LINKCOM uses "TEMPFILE", I get a link failure because
>>>>>> armlink.exe
>>>>>> > checks if --cpu is specified on the command line, so it is not
>>>>>> > reasonable for "TEMPFILE" to put all options to a temporary file.
>>>>>>
>>>>>> Right... thus my suggestion - well, let's call it "musing" rather than
>>>>>> "suggestion" because this might not work at all:
>>>>>>
>>>>>> > For ninja it currently looks like this:
>>>>>> >
>>>>>> > env["LINKCOM"] = '${TEMPFILE("$LINK $LINKFLAGS /OUT:$TARGET.windows
>>>>>> > $_LIBDIRFLAGS $_LIBFLAGS $_PDB $SOURCES.windows", "$LINKCOMSTR")}'
>>>>>> >
>>>>>> > env["SHLINKCOM"] = '${TEMPFILE("$SHLINK $SHLINKFLAGS
>>>>>> $_SHLINK_TARGETS
>>>>>> > $_LIBDIRFLAGS $_LIBFLAGS $_PDB $_SHLINK_SOURCES", "$SHLINKCOMSTR")}'
>>>>>> >
>>>>>> > So I'm curious if moving $LINK $LINKFLAGS to the left of ${TEMPFILE
>>>>>> > would help...
>>>>>> _______________________________________________
>>>>>> Scons-users mailing list
>>>>>> Scons-users at scons.org
>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>
>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20220415/bc3f9d74/attachment-0001.htm>

[Scons-users] 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option

[Scons-users] 回复：回复：回复：回复：回复： Ninja compilation fails after using include path file instead of "-I" option