[Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
liruncong2018
liruncong2018 at qq.com
Fri Apr 22 23:07:22 EDT 2022
Hi,
I tested it after the update, still the same problem, I uploaded the build.ninja file again (after updating the ninja_integration_branch branch).
------------------ 原始邮件 ------------------
发件人: "Daniel Moody" <dmoody256 at gmail.com>;
发送时间: 2022年4月23日(星期六) 凌晨0:54
收件人: "liruncong2018"<liruncong2018 at qq.com>;
抄送: "Bill Deegan"<bill at baddogconsulting.com>;"scons-users"<scons-users at scons.org>;
主题: Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
I think I know why this is happening, Ninja will parse header dependencies via the "-MMD -MF" or "/showIncludes" options. Ninja must be told what format to parse from the output of these options. By default ninja uses MSVC format on windows, but you are using gcc and clang compilers on windows so msvc format is not correct.
I have added a new configuration variable to allow you to tell scons what to configure ninja for, before ninja tool is loaded via env.Tool('ninja') in the SConstruct, you must set the variable NINJA_DEPFILE_PARSE_FORMAT, so it will look like this in the SContruct:
env['NINJA_DEPFILE_PARSE_FORMAT'] = 'clang'
env.Tool('Ninja')
Please pull the latest changes from https://github.com/dmoody256/scons/tree/ninja_integration_branch
On Fri, Apr 22, 2022 at 10:06 AM liruncong2018 <liruncong2018 at qq.com> wrote:
Hello Daniel Moody,
AX.h/AX.cpp
------------------ 原始邮件 ------------------
发件人: "Daniel Moody" <dmoody256 at gmail.com>;
发送时间: 2022年4月22日(星期五) 晚上10:56
收件人: "liruncong2018"<liruncong2018 at qq.com>;
抄送: "Bill Deegan"<bill at baddogconsulting.com>;"scons-users"<scons-users at scons.org>;
主题: Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
yes can you give the name of the cpp file and header file that were involved in the issue?
On Fri, Apr 22, 2022 at 9:46 AM liruncong2018 <liruncong2018 at qq.com> wrote:
Hi,
Does the modification of the .h recompile correctly with only scons build? -- Yes. It can be determined that the problem occurs after ninja is enabled.
Can you tell me which cpp file and what header file? Can you put your most recent branch in github so I can inspect the files? -- I have tried to reproduce the problem in a test project, but without success.
Do you have all the paths to your header files in the CPPPATH ? -- Yes. It is normal when ninja is not enabled.
I uploaded build.ninja, don't know if it helps in analysis.
------------------ 原始邮件 ------------------
发件人: "Bill Deegan" <bill at baddogconsulting.com>;
发送时间: 2022年4月22日(星期五) 凌晨0:04
收件人: "Daniel Moody"<dmoody256 at gmail.com>;
抄送: "liruncong2018"<liruncong2018 at qq.com>;"scons-users"<scons-users at scons.org>;
主题: Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
Do you have all the paths to your header files in the CPPPATH ?
On Thu, Apr 21, 2022 at 4:42 PM Daniel Moody <dmoody256 at gmail.com> wrote:
Does the modification of the .h recompile correctly with only scons build? Can you tell me which cpp file and what header file? Can you put your most recent branch in github so I can inspect the files?
On Thu, Apr 21, 2022 at 8:52 AM liruncong2018 <liruncong2018 at qq.com> wrote:
Hi,
Now after enabling the ninja function, if we modify the .cpp file, then only this file will be compiled and then linked, which is correct.
But if the modification is a .h file, the source file containing the file will not be compiled, and the compilation will take no action.
------------------ 原始邮件 ------------------
发件人: "liruncong2018" <liruncong2018 at qq.com>;
发送时间: 2022年4月21日(星期四) 中午11:48
收件人: "Daniel Moody"<dmoody256 at gmail.com>;
抄送: "Bill Deegan"<bill at baddogconsulting.com>;"scons-users"<scons-users at scons.org>;
主题: 回复： [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
Hello Daniel Moody,
Solved, thanks!!!
------------------ 原始邮件 ------------------
发件人: "Daniel Moody" <dmoody256 at gmail.com>;
发送时间: 2022年4月21日(星期四) 上午10:41
收件人: "liruncong2018"<liruncong2018 at qq.com>;
抄送: "Bill Deegan"<bill at baddogconsulting.com>;"scons-users"<scons-users at scons.org>;
主题: Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
Hello liruncong,
I will briefly explain scanners in your situation.
Every builder (ex. Object, SharedObject, Program, SharedLibrary, Textfile) has a target_scanner and source_scanner attribute. This attribute is the scanner which should run to find dependencies for either the targets or the sources, corresponding to target_scanner and source_scanner.
The example scanner in your previous email is the scanner being applied to the Object and SharedObject builders ( SCons.Tool.createObjBuilders(env) will return these two builders). These builders are what do the compilation (.c/.cpp -> .o), in this case the target will be the output .o file from compilation so we are interested in the target_scanner. When you apply this scanner to those Object and SharedObject builders, it will be run for every separate compilation and return the list of dependencies that should be built first before scons will proceed with that compilation. linkObjsFile is a list of object files and you should not pass object files to the compilation command. Therefore there is no reason .o files should depend on this file because it should not be in the command line and therefore it should not be in the scanner for object files.
I have already said linkObjsFile does not belong in the scanner for object files. linkObjsFile should be on the command line which is linking a program or library binary and therefore should be in a scanner on the Program, StaticLibrary or SharedLibrary builder. The Program builder already has a target scanner it uses to find the library files which are used on the command line, so therefore you can not simply overwrite the existing scanner (unless you take care to replicate the existing logic in your new scanner). You can chain the scanner, so that when your scanner is called it calls the existing scanner, then adds some more dependencies to the return list. I have sent you an example of how to chain the scanner in your case for the Program builder.
env.Depends is meant for special case use, because it deals with specific nodes. Scanners are meant to determine dependencies for all cases of building sources to targets.
On Wed, Apr 20, 2022 at 8:51 PM liruncong2018 <liruncong2018 at qq.com> wrote:
Hi,
After adding linkObjsFile to the scanner, adding a .cpp file will cause the compilation of all files, not just the new file. linkObjsFile should only be added to the target's dependencies, how to change it? Compared with env.Depends, scanner is more difficult to understand.
——————————————————————————————————————————————————————————
def _add_scanner(builder):
def new_scanner(node, env, path):
returndefineOptsFile + incPathsFile + exeIncPathsFile + linkObjsFile + env["_H_Files"] + [cPch] + [cppPch]
builder.builder.target_scanner = SCons.Scanner.Scanner(
function=new_scanner,
path_function=[],
)
forobject_builder in SCons.Tool.createObjBuilders(env):
_add_scanner(object_builder)
——————————————————————————————————————————————————————————
------------------ 原始邮件 ------------------
发件人: "liruncong2018" <liruncong2018 at qq.com>;
发送时间: 2022年4月19日(星期二) 晚上11:16
收件人: "Daniel Moody"<dmoody256 at gmail.com>;
抄送: "Bill Deegan"<bill at baddogconsulting.com>;"scons-users"<scons-users at scons.org>;
主题: 回复： [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
Hello Daniel Moody,
# Daniel Moody: This text file is only dependent on the string list, so only if the
# objsList changes would the Textfile be regenerated
At present, the sorted function is used to solve the problem that the obj list may be different each time,
# Daniel Moody: Setting the LINKCOM or any of the COM variables directly is a very aggressive
# approach and should not be used in most situations.
# Instead you should add the new flags to LINKFLAGS like:
# env.Append(LINKFLAGS=["--via=%s" % (linkObjsFile[0].path.replace("\\", "/"))])
This can't be changed, I need to remove $SOURCE from LINKCOM and replace it with --via=file. Is currently changed to: env["LINKCOM"] = "$LINK -o $TARGET $LINKFLAGS $__RPATH $_LIBDIRFLAGS $_LIBFLAGS --via=%s" % (linkObjsFile[0].path.replace("\\", "/"))
# Daniel Moody: This scanner is for object files only, here I see the use of exeIncPathsFile and
# linkObjsFile which I assume are not meant to be used with object file compile commands. I
# think you will want to make a scanner for the Program Builder. The program builder already has
# a scanner so you will need to chain the scanners like this:
#
# def new_program_scanner(node, env, path, argument):
# dependencies = argument(node, env, path)
# dependencies += exeIncPathsFile + linkObjsFile
# return dependencies
# env['BUILDERS']['Program'].target_scanner = SCons.Scanner.Scanner(
# function=new_program_scanner,
# path_function=env['BUILDERS']['Program'].target_scanner.path_function,
# argument=env['BUILDERS']['Program'].target_scanner
# )
I still don't quite understand this, I may have to check the scons manual.
# Daniel Moody: I think you are trying to create dependencies by passing the response files as a source here?
# I think instead you should use the scanner I recommended in the above comment.
# also I do not know what rtconfig.TARGET is or why it needs to depend on the exeTarget
rtconfig.TARGET and exeTarget are actually two separate targets, but they share some files. exeTarget is relatively simple, compiling directly from .cpp file to .exe file (compiling and linking with one command), avoiding the need to prepare an additional environment.
I don't understand the new_program_scanner above, and I don't know how to change it for the time being. However, the last env.Depends does not seem to affect the ninja function, at least the current test results.
Hello,
After the update, there is no problem with the ninja function for the time being.Thanks!
Hello,
After I updated, the behavior of enabling ninja and not enabling ninja is now consistent, that is, recompiling every time. This is a problem caused by linkObjsFile being created every time, I need to solve this problem first, and then continue to test down.
------------------ 原始邮件 ------------------
发件人: "Daniel Moody" <dmoody256 at gmail.com>;
发送时间: 2022年4月19日(星期二) 凌晨4:05
收件人: "liruncong2018"<liruncong2018 at qq.com>;
抄送: "Bill Deegan"<bill at baddogconsulting.com>;"scons-users"<scons-users at scons.org>;
主题: Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
If you use the "generate-ninja" command line target does it execute run_ninja_env.bat?
I pushed another update to the branch if you could try the latest?
On Mon, Apr 18, 2022 at 10:59 AM liruncong2018 <liruncong2018 at qq.com> wrote:
Hello,
After the update, the phenomenon is exactly the same, the modified file is still not compiled, and the scons still does not call run_ninja_env.bat.
I haven't had time to confirm your suggestions later, I'll take a look tomorrow night, thanks!
------------------ 原始邮件 ------------------
发件人: "Daniel Moody" <dmoody256 at gmail.com>;
发送时间: 2022年4月18日(星期一) 晚上11:09
收件人: "liruncong2018"<liruncong2018 at qq.com>;
抄送: "Bill Deegan"<bill at baddogconsulting.com>;"scons-users"<scons-users at scons.org>;
主题: Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
Hello,
Thanks for reporting the issue from ninja, I have updated the branch with a small fix for that case.
Regarding the scons issues, I added comments inline about the code and some of the issues I see:
compileIncPath = os.path.normpath(os.path.join(rtconfig.EXEC_PATH, "..", "include"))
cppPathList = [os.path.join(compileIncPath, "libcxx"), compileIncPath] + env["CPPPATH"]
cppPathStr = "-I" + "\n-I".join(cppPathList).replace("\\", "/")
incPathsFile = env.Textfile(
"build/__cpp_path.txt",
cppPathStr
)
env["_CPPINCFLAGS"] = "@%s" % (incPathsFile[0].path.replace("\\", "/"))
exeIncPath = []
for p in env["CPPPATH"]:
p = p.replace("\\", "/")
if p.find("compilers") == -1 and not re.search('components/net', p):
exeIncPath += [p]
exeCppPathStr = "-I" + "\n-I".join(exeIncPath).replace("\\", "/")
exeIncPathsFile = env.Textfile(
"build/__exe_cpp_path.txt",
exeCppPathStr
)
env["_ExeIncPathsFile"] = exeIncPathsFile[0].path.replace("\\", "/")
objsList = []
for obj in objs:
(fileName, ext) = os.path.splitext(obj.path)
if ext in env["CPPSUFFIXES"] or ext == ".o":
objsList += [fileName.replace("\\", "/") + ".o"];
# Daniel Moody: This text file is only dependent on the string list, so only if the
# objsList changes would the Textfile be regenerated
linkObjsFile = env.Textfile(
"build/__link_objects.txt",
"\n".join(objsList)
)
# Daniel Moody: Setting the LINKCOM or any of the COM variables directly is a very aggressive
# approach and should not be used in most situations.
# Instead you should add the new flags to LINKFLAGS like:
# env.Append(LINKFLAGS=["--via=%s" % (linkObjsFile[0].path.replace("\\", "/"))])
env["LINKCOM"] = "$LINK -o $TARGET $LINKFLAGS --via=%s" % (linkObjsFile[0].path.replace("\\", "/"))
defineOptsFile = env.Textfile(
"build/__define_options.txt",
"\n".join(rtconfig.rtconfigObj.DefineOptions)
)
incFile = os.path.join(rtconfig.KND_ROOT, "include", "basicdef.h")
cPch = os.path.join(rtconfig.BSP_ROOT, "build", os.path.basename(incFile)) + ".c.pch"
cppPch = os.path.join(rtconfig.BSP_ROOT, "build", os.path.basename(incFile)) + ".cpp.pch"
env.Command(
cPch,
[incFile, incPathsFile, defineOptsFile],
["$CXX -x c-header ${SOURCES[0]} -o $TARGET $CCFLAGS $CFLAGS $_CPPINCFLAGS @$_DefineOptsFile"]
)
env.Command(
cppPch,
[incFile, incPathsFile, defineOptsFile],
["$CXX -x c++-header ${SOURCES[0]} -o $TARGET $CCFLAGS $CXXFLAGS $_CPPINCFLAGS @$_DefineOptsFile"]
)
env["_C_INC_FLAG"] = "-include %s" % (incFile)
env["_CXX_INC_FLAG"] = "-include %s" % (incFile)
env["_C_PCH_FLAG"] = "-include-pch %s" % (cPch)
env["_CXX_PCH_FLAG"] = "-include-pch %s" % (cppPch)
# Daniel Moody: This scanner is for object files only, here I see the use of exeIncPathsFile and
# linkObjsFile which I assume are not meant to be used with object file compile commands. I
# think you will want to make a scanner for the Program Builder. The program builder already has
# a scanner so you will need to chain the scanners like this:
#
# def new_program_scanner(node, env, path, argument):
# dependencies = argument(node, env, path)
# dependencies += exeIncPathsFile + linkObjsFile
# return dependencies
# env['BUILDERS']['Program'].target_scanner = SCons.Scanner.Scanner(
# function=new_program_scanner,
# path_function=env['BUILDERS']['Program'].target_scanner.path_function,
# argument=env['BUILDERS']['Program'].target_scanner
# )
def _add_scanner(builder):
def new_scanner(node, env, path):
return defineOptsFile + incPathsFile + exeIncPathsFile + linkObjsFile + env["_H_Files"] + [cPch] + [cppPch]
builder.builder.target_scanner = SCons.Scanner.Scanner(
function=new_scanner,
path_function=[],
)
for object_builder in SCons.Tool.createObjBuilders(env):
_add_scanner(object_builder)
env["_DefineOptsFile"] = defineOptsFile
env["CCCOM"] += " $_C_INC_FLAG $_C_PCH_FLAG @$_DefineOptsFile"
env["CXXCOM"] += " $_CXX_INC_FLAG $_CXX_PCH_FLAG @$_DefineOptsFile"
opts = [
"-Wno-invalid-source-encoding",
"-Wno-invalid-offsetof",
"-Wformat",
"-Wuninitialized",
"-Wfloat-conversion",
"-Wno-unused-variable",
"-Wno-int-to-void-pointer-cast",
"-Wno-deprecated-declarations",
"-Wno-int-to-pointer-cast",
"-std=gnu++17",
"-Wno-c++2a-extensions", # Synonym for -Wno-c++20-extensions.
"-Wno-gnu-string-literal-operator-template", # string literal operator templates are a GNU extension
"-D__MSVC_INLINE_ASM=",
"-DWIN32"
]
# Daniel Moody: I think you are trying to create dependencies by passing the response files as a source here?
# I think instead you should use the scanner I recommended in the above comment.
# also I do not know what rtconfig.TARGET is or why it needs to depend on the exeTarget
for exeTarget, srcList in env["_ExeTarget"]:
env.Command(
exeTarget,
[srcList, exeIncPathsFile, defineOptsFile],
[
"clang++.exe -o $TARGET ${SOURCES[0]} @$_ExeIncPathsFile $_CXX_INC_FLAG @$_DefineOptsFile %s" % (" ".join(opts))
]
)
env.Depends(rtconfig.TARGET, exeTarget)
On Mon, Apr 18, 2022 at 8:26 AM liruncong2018 <liruncong2018 at qq.com> wrote:
Hi,
I get the following error after compiling with branch: "https://github.com/dmoody256/scons/tree/ninja_integration_branch"
···
ninja: error: D:\work\current\master\bsp\am335x-knd\apps\cnc\build.ninja:268806: unknown target 'D'
default D:\work\current\master\knd\libs\CncSys\K2000PRJ\LimitTime\LimitT...
^ near here
scons: *** [build.ninja] CalledProcessError : Command 'ninja' returned non-zero exit status 1.[error]
Traceback (most recent call last):
···
"D:\work\current\***" should it be "D$:\work\current\***" ?
In addition, I found that when ninja is not enabled, scons will all be recompiled when compiled again, which is equivalent to the first compilation.
From the log, "__link_objects.txt" is generated again, which will cause all to be recompiled.
Below is my code for this part, I don't know if there is a problem. At the same time, the last env.Depends inside I don't know how to use scanner instead?
···
compileIncPath = os.path.normpath(os.path.join(rtconfig.EXEC_PATH, "..", "include"))
cppPathList = [os.path.join(compileIncPath, "libcxx"), compileIncPath] + env["CPPPATH"]
cppPathStr = "-I" + "\n-I".join(cppPathList).replace("\\", "/")
incPathsFile = env.Textfile(
"build/__cpp_path.txt",
cppPathStr
)
env["_CPPINCFLAGS"] = "@%s" % (incPathsFile[0].path.replace("\\", "/"))
exeIncPath = []
for p in env["CPPPATH"]:
p = p.replace("\\", "/")
if p.find("compilers") == -1 and not re.search('components/net', p):
exeIncPath += [p]
exeCppPathStr = "-I" + "\n-I".join(exeIncPath).replace("\\", "/")
exeIncPathsFile = env.Textfile(
"build/__exe_cpp_path.txt",
exeCppPathStr
)
env["_ExeIncPathsFile"] = exeIncPathsFile[0].path.replace("\\", "/")
objsList = []
for obj in objs:
(fileName, ext) = os.path.splitext(obj.path)
if ext in env["CPPSUFFIXES"] or ext == ".o":
objsList += [fileName.replace("\\", "/") + ".o"];
linkObjsFile = env.Textfile(
"build/__link_objects.txt",
"\n".join(objsList)
)
env["LINKCOM"] = "$LINK -o $TARGET $LINKFLAGS --via=%s" % (linkObjsFile[0].path.replace("\\", "/"))
defineOptsFile = env.Textfile(
"build/__define_options.txt",
"\n".join(rtconfig.rtconfigObj.DefineOptions)
)
incFile = os.path.join(rtconfig.KND_ROOT, "include", "basicdef.h")
cPch = os.path.join(rtconfig.BSP_ROOT, "build", os.path.basename(incFile)) + ".c.pch"
cppPch = os.path.join(rtconfig.BSP_ROOT, "build", os.path.basename(incFile)) + ".cpp.pch"
env.Command(
cPch,
[incFile, incPathsFile, defineOptsFile],
["$CXX -x c-header ${SOURCES[0]} -o $TARGET $CCFLAGS $CFLAGS $_CPPINCFLAGS @$_DefineOptsFile"]
)
env.Command(
cppPch,
[incFile, incPathsFile, defineOptsFile],
["$CXX -x c++-header ${SOURCES[0]} -o $TARGET $CCFLAGS $CXXFLAGS $_CPPINCFLAGS @$_DefineOptsFile"]
)
env["_C_INC_FLAG"] = "-include %s" % (incFile)
env["_CXX_INC_FLAG"] = "-include %s" % (incFile)
env["_C_PCH_FLAG"] = "-include-pch %s" % (cPch)
env["_CXX_PCH_FLAG"] = "-include-pch %s" % (cppPch)
def _add_scanner(builder):
def new_scanner(node, env, path):
return defineOptsFile + incPathsFile + exeIncPathsFile + linkObjsFile + env["_H_Files"] + [cPch] + [cppPch]
builder.builder.target_scanner = SCons.Scanner.Scanner(
function=new_scanner,
path_function=[],
)
for object_builder in SCons.Tool.createObjBuilders(env):
_add_scanner(object_builder)
env["_DefineOptsFile"] = defineOptsFile
env["CCCOM"] += " $_C_INC_FLAG $_C_PCH_FLAG @$_DefineOptsFile"
env["CXXCOM"] += " $_CXX_INC_FLAG $_CXX_PCH_FLAG @$_DefineOptsFile"
opts = [
"-Wno-invalid-source-encoding",
"-Wno-invalid-offsetof",
"-Wformat",
"-Wuninitialized",
"-Wfloat-conversion",
"-Wno-unused-variable",
"-Wno-int-to-void-pointer-cast",
"-Wno-deprecated-declarations",
"-Wno-int-to-pointer-cast",
"-std=gnu++17",
"-Wno-c++2a-extensions", # Synonym for -Wno-c++20-extensions.
"-Wno-gnu-string-literal-operator-template", # string literal operator templates are a GNU extension
"-D__MSVC_INLINE_ASM=",
"-DWIN32"
]
for exeTarget, srcList in env["_ExeTarget"]:
env.Command(
exeTarget,
[srcList, exeIncPathsFile, defineOptsFile],
[
"clang++.exe -o $TARGET ${SOURCES[0]} @$_ExeIncPathsFile $_CXX_INC_FLAG @$_DefineOptsFile %s" % (" ".join(opts))
]
)
env.Depends(rtconfig.TARGET, exeTarget)
···
------------------ 原始邮件 ------------------
发件人: "Daniel Moody" <dmoody256 at gmail.com>;
发送时间: 2022年4月18日(星期一) 下午3:36
收件人: "liruncong2018"<liruncong2018 at qq.com>;
抄送: "Bill Deegan"<bill at baddogconsulting.com>;"scons-users"<scons-users at scons.org>;
主题: Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
Hello liruncong,
I have a test branch for the issues you described where re-running scons is not re-compiling, could you please try it out?: https://github.com/dmoody256/scons/tree/ninja_integration_branch
On Sun, Apr 17, 2022 at 2:29 AM liruncong2018 <liruncong2018 at qq.com> wrote:
If execute the command "scons -j%NUMBER_OF_PROCESSORS% generate-ninja --verbose --experimental=ninja", it will be completely recompiled.
If execute the command again, then it will be completely recompiled.
Each time is equivalent to the first compilation.
------------------ 原始邮件 ------------------
发件人: "Daniel Moody" <dmoody256 at gmail.com>;
发送时间: 2022年4月17日(星期天) 中午1:52
收件人: "liruncong2018"<liruncong2018 at qq.com>;
抄送: "Bill Deegan"<bill at baddogconsulting.com>;"SCons users mailing list"<scons-users at scons.org>;
主题: Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
If you use the command line target 'generate-ninja' for the scons command, does it still fail to build and execute ninja on the second run?
On Sat, Apr 16, 2022 at 10:36 PM liruncong2018 <liruncong2018 at qq.com> wrote:
Great, looking forward to your PR!
------------------ 原始邮件 ------------------
发件人: "Daniel Moody" <dmoody256 at gmail.com>;
发送时间: 2022年4月17日(星期天) 中午11:31
收件人: "liruncong2018"<liruncong2018 at qq.com>;
抄送: "Bill Deegan"<bill at baddogconsulting.com>;"SCons users mailing list"<scons-users at scons.org>;
主题: Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
Okay that indicates that scons is not deciding to build and execute the ninja file, probably because nothing has changed in the build scripts when you only change the source file. This is a bug, even if the ninja file is not regened, scons should execute it for you. I will make a PR for this issue. For now you will need to use the run_ninja_env.bat.
On Sat, Apr 16, 2022 at 9:54 PM liruncong2018 <liruncong2018 at qq.com> wrote:
Hi,
When I just execute run_ninja_env.bat, it shows compile error. It seems that there is no problem with ninja, the problem should be that scons does not call run_ninja_env.bat to execute. Now how do I diagnose this problem with scons?
------------------ 原始邮件 ------------------
发件人: "Daniel Moody" <dmoody256 at gmail.com>;
发送时间: 2022年4月17日(星期天) 上午10:28
收件人: "liruncong2018"<liruncong2018 at qq.com>;
抄送: "Bill Deegan"<bill at baddogconsulting.com>;"SCons users mailing list"<scons-users at scons.org>;
主题: Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
When I test with the repo https://github.com/liruncong/NinJaTest for bsp/beaglebone build using the changes from here: https://github.com/SCons/scons/pull/4133, and I put an 'a' in the src/irq.c file, I see the file error when I rerun the build.
Could you specify what file you changed?
Also on subsequent runs, you can just run_ninja_env.bat, it will skip scons regeneration, but if you want scons to regen it should work also.
On Sat, Apr 16, 2022 at 8:46 PM liruncong2018 <liruncong2018 at qq.com> wrote:
Hi,
After the first compilation is successful, modify the file, expect the compilation to fail, but compile again without error message.
The following is the log after the command("scons -j%NUMBER_OF_PROCESSORS% --verbose --experimental=ninja") is executed:
______________________________________
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
scons: done building targets.
______________________________________
I have attached the compressed "build.ninja" file in the attachment.
------------------ 原始邮件 ------------------
发件人: "Daniel Moody" <dmoody256 at gmail.com>;
发送时间: 2022年4月17日(星期天) 凌晨1:53
收件人: "Bill Deegan"<bill at baddogconsulting.com>;
抄送: "SCons users mailing list"<scons-users at scons.org>;"liruncong2018"<liruncong2018 at qq.com>;
主题: Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
Also i realized that link_objects.txt is probably not meant for the object builders. I assume this file is intended for linking the libraries and programs? In that case you will want a scanner for those builders instead. Do you have error messages you can show, it should help to see what your issue is.
On Sat, Apr 16, 2022, 10:58 AM Bill Deegan <bill at baddogconsulting.com> wrote:
try bzip'ing the ninja.build file. That should make it considerably smaller.
On Sat, Apr 16, 2022 at 4:37 PM liruncong2018 via Scons-users <scons-users at scons.org> wrote:
Hi,
Still the same problem. I want to send "build.ninja" as an attachment, but the email always bounces, what information can I print so I can find the problem?
------------------ 原始邮件 ------------------
发件人: "Daniel Moody" <dmoody256 at gmail.com>;
发送时间: 2022年4月16日(星期六) 凌晨0:29
收件人: "liruncong2018"<liruncong2018 at qq.com>;
抄送: "SCons users mailing list"<scons-users at scons.org>;
主题: Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
Here's the same code but better color scheme:
incPathsFile = env.Textfile(
"build/__cpp_path.txt",
cppPathStr
)
defineOptsFile = env.Textfile(
"build/__define_options.txt",
"\n".join(rtconfig.rtconfigObj.DefineOptions)
)
linkObjsFile = env.Textfile(
"build/__link_objects.txt",
"\n".join(objsList)
)
def _add_scanner(builder):
def new_scanner(node, env, path):
return incPathsFile + defineOptsFile + linkObjsFile
builder.builder.target_scanner = SCons.Scanner.Scanner(
function=new_scanner,
path_function=[],
)
for object_builder in SCons.Tool.createObjBuilders(env):
_add_scanner(object_builder)
On Fri, Apr 15, 2022 at 10:50 AM Daniel Moody <dmoody256 at gmail.com> wrote:
hello liruncong,
If you want to add multiple @file dependencies via the scanner do them all in the same scanner. Like I said you can only apply one scanner, so in this case you are overwriting each previous scanner. The scanner should return a list of files in which to set dependencies for the current node.
The path function is not used in this case, and I just left the CPPPATH from the default SourceFileScanner I copied the scanner code from. It does have to be something besides None I believe though, or scons won't scan. The correct way this should be used in this case is you should place the @file in a known list of directories or just a single directory, and then pass a list of directories or a list containing the single directory, and the scanner should find the files by extension in the list of directories. This is the fundamental working of scanners if you want to read more: https://scons.org/doc/production/HTML/scons-user/ch20.html, however you could also use a dummy list if there is no dynamic component here.
Here is an updated example based on the new info you provided:
incPathsFile = env.Textfile(
"build/__cpp_path.txt",
cppPathStr
)
defineOptsFile = env.Textfile(
"build/__define_options.txt",
"\n".join(rtconfig.rtconfigObj.DefineOptions)
)
linkObjsFile = env.Textfile(
"build/__link_objects.txt",
"\n".join(objsList)
)
def _add_scanner(builder):
def new_scanner(node, env, path):
return incPathsFile + defineOptsFile + linkObjsFile
builder.builder.target_scanner = SCons.Scanner.Scanner(
function=new_scanner,
path_function=[],
)
for object_builder in SCons.Tool.createObjBuilders(env):
_add_scanner(object_builder)
On Fri, Apr 15, 2022 at 9:58 AM liruncong2018 <liruncong2018 at qq.com> wrote:
Can you give me more information? What source file? Does the compilation fail the first time? Does it recompile successfully with only scons building? Can you push your latest code so I can see it in your repo?
-- I don't know how to reproduce this problem in the git test project, I will try to see if it can be reproduced.
Also each builder can only hold one target or source scanner, but you can chain the scanner by calling other scanners from within the single scanner. Are you assigning multiple scanners to the same builder?
-- The following are some scanners, I don't quite understand the path_function, especially the second parameter of the last scanner (linkObjsFile).
————————————————————————————————————————
incPathsFile = env.Textfile(
"build/__cpp_path.txt",
cppPathStr
)
def _add_scanner(builder):
def new_scanner(node, env, path):
return incPathsFile
builder.builder.target_scanner = SCons.Scanner.Scanner(
function=new_scanner,
path_function=SCons.Script.FindPathDirs('CPPPATH'),
)
for object_builder in SCons.Tool.createObjBuilders(env):
_add_scanner(object_builder)
defineOptsFile = env.Textfile(
"build/__define_options.txt",
"\n".join(rtconfig.rtconfigObj.DefineOptions)
)
def _add_scanner(builder):
def new_scanner(node, env, path):
return defineOptsFile
builder.builder.target_scanner = SCons.Scanner.Scanner(
function=new_scanner,
path_function=SCons.Script.FindPathDirs('CPPPATH'),
)
for object_builder in SCons.Tool.createObjBuilders(env):
_add_scanner(object_builder)
linkObjsFile = env.Textfile(
"build/__link_objects.txt",
"\n".join(objsList)
)
def _add_scanner(builder):
def new_scanner(node, env, path):
return linkObjsFile
builder.builder.target_scanner = SCons.Scanner.Scanner(
function=new_scanner,
path_function=SCons.Script.FindPathDirs('LIBPATH'),
)
for object_builder in SCons.Tool.createObjBuilders(env):
_add_scanner(object_builder)
————————————————————————————————————————————————
------------------ 原始邮件 ------------------
发件人: "Daniel Moody" <dmoody256 at gmail.com>;
发送时间: 2022年4月14日(星期四) 晚上11:36
收件人: "liruncong2018"<liruncong2018 at qq.com>;
抄送: "SCons users mailing list"<scons-users at scons.org>;
主题: Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
Can you give me more information? What source file? Does the compilation fail the first time? Does it recompile successfully with only scons building? Can you push your latest code so I can see it in your repo?
Also each builder can only hold one target or source scanner, but you can chain the scanner by calling other scanners from within the single scanner. Are you assigning multiple scanners to the same builder?
On Thu, Apr 14, 2022 at 10:17 AM liruncong2018 <liruncong2018 at qq.com> wrote:
Hello Daniel Moody,
Thanks a lot for the example you gave. According to your example, I added 4 scanners, which can indeed be compiled and linked.
But when I edit the source file, I deliberately put "a" at the end of the .cpp file and expect the compilation to fail. But when I compile again, nothing compiles.
How to diagnose this problem?
------------------ 原始邮件 ------------------
发件人: "Daniel Moody" <dmoody256 at gmail.com>;
发送时间: 2022年4月14日(星期四) 凌晨2:27
收件人: "liruncong2018"<liruncong2018 at qq.com>;
抄送: "SCons users mailing list"<scons-users at scons.org>;
主题: Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
I added an example how to hook up the dependencies:
https://github.com/liruncong/NinJaTest/pull/1/files
On Wed, Apr 13, 2022 at 10:01 AM liruncong2018 <liruncong2018 at qq.com> wrote:
Hi,
I generate some option files via env.Textfile ("build/__cpp_path.txt", "build/__define_options.txt", "build/__link_objects.txt", "build/__exe_cpp_path.txt") and make objs depend on corresponding files ,for example:
env.Depends(objs, ["build/__cpp_path.txt", "build/__define_options.txt" ])
env.Depends(target, "build/__link_objects.txt")
But after enabling ninja, these manually added dependencies are ignored, which causes ninja to fail to compile.
How to deal with this?
------------------ 原始邮件 ------------------
发件人: "Daniel Moody" <dmoody256 at gmail.com>;
发送时间: 2022年4月13日(星期三) 凌晨0:02
收件人: "SCons users mailing list"<scons-users at scons.org>;
抄送: "liruncong2018"<liruncong2018 at qq.com>;
主题: Re: [Scons-users] 回复： 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
Hello, that PR is meant to support shared @file with ninja, not intended for any improvements to TEMPFILE. Please test without TEMPFILE.
On Tue, Apr 12, 2022 at 10:52 AM liruncong2018 via Scons-users <scons-users at scons.org> wrote:
Hi
I tested "https://github.com/SCons/scons/pull/4133", after enabling ningja, except for the link failure, .c/.cpp/.S are successfully compiled into .o, the total time is 6:48 .
This time is much greater than the previous time of 3:45 when ninja was not enabled and the option file was shared using @file.
After using TEMPFILE instead of @file, the compilation time is basically doubled whether ninja is enabled or not.
The TEMPFILE function should be improved to use shared files to prevent the creation of a large number of temporary files.
------------------ 原始邮件 ------------------
发件人: "SCons users mailing list" <dmoody256 at gmail.com>;
发送时间: 2022年4月11日(星期一) 晚上11:43
收件人: "SCons users mailing list"<scons-users at scons.org>;
主题: Re: [Scons-users] 回复： 回复： 回复： 回复： Ninja compilation fails after using include path file instead of "-I" option
I have a potential fix to the ninja tool so that it can decide to use response files or not. This should allow you to use your own shared response file.
https://github.com/SCons/scons/pull/4133
On Mon, Apr 11, 2022 at 10:01 AM Mats Wichmann <mats at wichmann.us> wrote:
On 4/11/22 08:57, liruncong2018 via Scons-users wrote:
> Hi,
> Even using msys2 does not solve all problems. The 32K limit solves my
> compilation problems (.cpp -> .o) but not linking (.o -> target) because
> the link command line has around 137K (about 2500 .o files). So
> compilers such as gcc/clang/armclang will support "@file".
> Currently, scons needs to set up Depends when using @file. This
> dependency suggestion is still directly supported by scons, and users do
> not need to write dependencies. It should also be supported when ninja
> is enabled.
> When LINKCOM uses "TEMPFILE", I get a link failure because armlink.exe
> checks if --cpu is specified on the command line, so it is not
> reasonable for "TEMPFILE" to put all options to a temporary file.
Right... thus my suggestion - well, let's call it "musing" rather than
"suggestion" because this might not work at all:
> For ninja it currently looks like this:
>
> env["LINKCOM"] = '${TEMPFILE("$LINK $LINKFLAGS /OUT:$TARGET.windows
> $_LIBDIRFLAGS $_LIBFLAGS $_PDB $SOURCES.windows", "$LINKCOMSTR")}'
>
> env["SHLINKCOM"] = '${TEMPFILE("$SHLINK $SHLINKFLAGS $_SHLINK_TARGETS
> $_LIBDIRFLAGS $_LIBFLAGS $_PDB $_SHLINK_SOURCES", "$SHLINKCOMSTR")}'
>
> So I'm curious if moving $LINK $LINKFLAGS to the left of ${TEMPFILE
> would help...
_______________________________________________
Scons-users mailing list
Scons-users at scons.org
https://pairlist4.pair.net/mailman/listinfo/scons-users
_______________________________________________
Scons-users mailing list
Scons-users at scons.org
https://pairlist4.pair.net/mailman/listinfo/scons-users
_______________________________________________
Scons-users mailing list
Scons-users at scons.org
https://pairlist4.pair.net/mailman/listinfo/scons-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20220423/ab03c101/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: build.ninja.bz2
Type: application/octet-stream
Size: 193831 bytes
Desc: not available
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20220423/ab03c101/attachment-0001.obj>
More information about the Scons-users
mailing list