[Scons-users] 回复： 回复： env.Textfile problem
liruncong
liruncong2018 at qq.com
Mon May 23 03:01:53 EDT 2022
Hello Daniel Moody,
echo_response_file will cause the command line command to be too long!
__cpp_path.txt is used to solve the problem of too long commands, but now after switching to env.Command, it is put on the command line again.
__________________________________________
def echo_response_file(respone_list):
return (f'@echo {respone_list[0]} > $TARGET & ' +
' '.join(f'echo {item} >> $TARGET &' for item in respone_list[1:-1]) +
f' echo {respone_list[-1]} >> $TARGET')
compileIncPath = os.path.normpath(os.path.join(rtconfig.EXEC_PATH, "..", "include"))
cppPathStr = [
"-I%s" % (os.path.join(compileIncPath, "libcxx").replace("\\", "/")),
"-I%s" % (compileIncPath.replace("\\", "/")),
]
cppPathStr += ['-I' + path.replace("\\", "/") for path in env["CPPPATH"]]
incPathsFile = env.Command("build/__cpp_path.txt",
[],
echo_response_file(cppPathStr)
)
__________________________________________
------------------ 原始邮件 ------------------
发件人: "liruncong" <liruncong2018 at qq.com>;
发送时间: 2022年5月23日(星期一) 下午2:47
收件人: "Daniel Moody"<dmoody256 at gmail.com>;
抄送: "scons-users"<scons-users at scons.org>;
主题: 回复： [Scons-users] 回复： env.Textfile problem
Hello Daniel Moody,
Thank you!
But in the future, the difficulty for users should be reduced, and it is difficult for users to solve these problems by themselves!
------------------ 原始邮件 ------------------
发件人: "Daniel Moody" <dmoody256 at gmail.com>;
发送时间: 2022年5月23日(星期一) 中午1:53
收件人: "liruncong"<liruncong2018 at qq.com>;
抄送: "scons-users"<scons-users at scons.org>;
主题: Re: [Scons-users] 回复： env.Textfile problem
Hello liruncong,
For issue number 1:
with scons only, I ran the build with --debug=explain and the output stated the order of the object files is changing every build. I fixed it with a sorted list like this:
@@ -816,7 +816,7 @@ def DoBuilding(target, objects):
for group in Projects:
local_group(group, objects)
- program = Env.Program(target, objects)
+ program = Env.Program(target, sorted(objects))
EndBuilding(target, program)
For issue number 2:
this is unavoidable. As I explained, ninja can not know if those objects need to be rebuilt because only scons can know this, so it must call to scons to check. This is the output you will see, ninja is asking scons if these files need to be built again because of some scons change. Ninja can not know this, so it must always ask. SCons will see that the files do not need to change and it should not touch the files if they are already built. Ninja will see after it is done asking scons that the files have not been touched also and it will know not to rebuild any files which depended on those files which it asked scons to build.
Alternatively, you do not need scons to build these files directly. If you convert the creation of the files to a command that ninja can execute on the command line then ninja will not need to callback to scons. I showed an example in the PR: https://github.com/liruncong/NinJaTest/pull/2
For issue number 3:
This is expected behavior, ninja "-n" option is dry-run which means it won't build anything, just test graph execution. I was using "-n" as an example of passing a ninja argument to ninja from scons.
On Wed, May 18, 2022 at 7:01 AM liruncong2018 <liruncong2018 at qq.com> wrote:
Hi,
After updated,
1) When ninja is not enabled, the last link is always executed when the compilation is repeated.
log:
___________________________________________
scons: Reading SConscript files ...
Newlib version:2.4.0
scons: done reading SConscript files.
scons: Building targets ...
scons: building associated VariantDir targets: build
arm-none-eabi-gcc -o rtthread-beaglebone.elf -Wall -march=armv7-a -mtune=cortex-a8 -ftree-vectorize -ffast-math -mfpu=vfpv3-d16 -mfloat-abi=softfp -Wl,--gc-sections,-Map=rtthread-beaglebone.map,-cref,-u,Reset_Handler -T beaglebone_ram.lds -lm -lc @build/__link_objects.txt
scons: done building targets.
___________________________________________
2) When ninja is enabled, when recompiling, it always prints something that doesn't seem to actually execute the steps
log:
___________________________________________
[1/56] Starting scons daemon...[2K
[2/56] Defer to SCons to build build\__cpp_path.txt[2K
[3/56] Defer to SCons to build build\__define_options.txt[2K
[4/56] Defer to SCons to build build\__link_objects.txt
scons: done building targets.
___________________________________________
Judging from the file time, these txt files(__cpp_path.txt/__define_options.txt/__link_objects.txt) are not actually generated.
The following is the log using NINJA_CMD_ARGS, printing the same message every time:
___________________________________________
D:\work\gitee\rt-thread_ninja\bsp\beaglebone>scons -j%NUMBER_OF_PROCESSORS% --implicit-cache NINJA_CMD_ARGS="-n" --verbose --experimental=ninja
scons: Reading SConscript files ...
Newlib version:2.4.0
scons: done reading SConscript files.
scons: Building targets ...
Generating: build.ninja...
No updates detected, build.ninja not regenerated.
Executing: run_ninja_env.bat -n
D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set SystemDrive=C:
D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set SystemRoot=C:\Windows
D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set TEMP=C:\Users\ARCHYI~1\AppData\Local\Temp
D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set TMP=C:\Users\ARCHYI~1\AppData\Local\Temp
D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set USERPROFILE=C:\Users\archyizimi
D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set COMSPEC=C:\Windows\system32\cmd.exe
D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set PATH=D:\MYBIN\env\tools\gnu_gcc\arm_gcc\mingw\bin;D:\MYBIN\scons-local-4.3.1\;D:\MYBIN;D:\MYBIN\Anaconda3\;C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\Llvm\bin;C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.32.31326\bin\HostX86\x86;C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\VC\VCPackages;C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow;C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer;C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Current\bin\Roslyn;C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Team Tools\Performance Tools;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools\;C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\\x86;C:\Program Files (x86)\Windows Kits\10\bin\\x86;C:\Program Files\Microsoft Visual Studio\2022\Enterprise\\MSBuild\Current\Bin\amd64;C:\Windows\Microsoft.NET\Framework\v4.0.30319;C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\;C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files\Microsoft SQL Server\150\Tools\Binn\;C:\Program Files\IDM Computer Solutions\UltraEdit;C:\Program Files\IDM Computer Solutions\UltraCompare;C:\Program Files\TortoiseGit\bin;C:\Program Files\dotnet\;C:\Program Files\Git\cmd;C:\Users\archyizimi\AppData\Local\Microsoft\WindowsApps;C:\Users\archyizimi\.dotnet\tools;C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin;C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja;C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\VC\Linux\bin\ConnectionManagerExe;c:\MinGW\bin
D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set PATHEXT=.COM;.EXE;.BAT;.CMD
D:\work\gitee\rt-thread_ninja\bsp\beaglebone>D:\MYBIN\Anaconda3\lib\site-packages\ninja\data\bin\ninja.exe -f D:\work\gitee\rt-thread_ninja\bsp\beaglebone\build.ninja -n
[1/56] Starting scons daemon... [2K
[2/56] Defer to SCons to build build\__cpp_path.txt [2K
[3/56] Defer to SCons to build build\__link_objects.txt [2K
[4/56] Defer to SCons to build build\__define_options.txt [2K
[5/56] Compiling build\applications\board.o [2K
[6/56] Compiling build\applications\main.o [2K
[7/56] Compiling build\drivers\gpio.o [2K
[8/56] Compiling build\kernel\components\dfs\src\dfs_posix.o [2K
[9/56] Compiling build\kernel\components\dfs\src\dfs_file.o [2K
[10/56] Compiling build\drivers\uart.o [2K
[11/56] Compiling build\kernel\components\dfs\filesystems\devfs\devfs.o [2K
[12/56] Compiling build\kernel\components\dfs\src\dfs.o [2K
[13/56] Compiling build\kernel\components\dfs\src\dfs_fs.o [2K
[14/56] Compiling build\kernel\components\drivers\ipc\pipe.o [2K
[15/56] Compiling build\kernel\components\drivers\ipc\completion.o [2K
[16/56] Compiling build\kernel\components\drivers\ipc\dataqueue.o [2K
[17/56] Compiling build\kernel\components\drivers\ipc\ringbuffer.o [2K
[18/56] Compiling build\kernel\components\drivers\ipc\ringblk_buf.o [2K
[19/56] Compiling build\kernel\components\drivers\ipc\waitqueue.o [2K
[20/56] Compiling build\kernel\components\finsh\msh.o [2K
[21/56] Compiling build\kernel\components\drivers\serial\serial.o [2K
[22/56] Compiling build\kernel\components\drivers\ipc\workqueue.o [2K
[23/56] Compiling build\kernel\components\drivers\misc\pin.o [2K
[24/56] Compiling build\kernel\components\finsh\cmd.o [2K
[25/56] Compiling build\kernel\components\libc\compilers\common\time.o [2K
[26/56] Compiling build\kernel\components\finsh\shell.o [2K
[27/56] Compiling build\kernel\components\libc\compilers\common\stdlib.o [2K
[28/56] Building build\kernel\libcpu\arm\am335x\context_gcc.o [2K
[29/56] Compiling build\kernel\components\libc\compilers\newlib\syscalls.o [2K
[30/56] Compiling build\kernel\components\finsh\msh_file.o [2K
[31/56] Building build\kernel\libcpu\arm\am335x\start_gcc.o [2K
[32/56] Building build\kernel\libcpu\arm\am335x\cp15_gcc.o [2K
[33/56] Compiling build\kernel\libcpu\arm\am335x\interrupt.o [2K
[34/56] Compiling build\kernel\libcpu\arm\am335x\cpu.o [2K
[35/56] Compiling build\kernel\libcpu\arm\am335x\stack.o [2K
[36/56] Compiling build\kernel\libcpu\arm\am335x\mmu.o [2K
[37/56] Compiling build\kernel\libcpu\arm\am335x\trap.o [2K
[38/56] Compiling build\kernel\src\clock.o [2K
[39/56] Compiling build\kernel\libcpu\arm\common\backtrace.o [2K
[40/56] Compiling build\kernel\libcpu\arm\common\div0.o [2K
[41/56] Compiling build\kernel\libcpu\arm\common\showmem.o [2K
[42/56] Building build\kernel\libcpu\arm\am335x\vector_gcc.o [2K
[43/56] Compiling build\kernel\src\device.o [2K
[44/56] Compiling build\kernel\src\components.o [2K
[45/56] Compiling build\kernel\src\idle.o [2K
[46/56] Compiling build\kernel\src\irq.o [2K
[47/56] Compiling build\kernel\src\ipc.o [2K
[48/56] Compiling build\kernel\src\kservice.o [2K
[49/56] Compiling build\kernel\src\mempool.o [2K
[50/56] Compiling build\kernel\src\object.o [2K
[51/56] Compiling build\kernel\src\thread.o [2K
[52/56] Compiling build\kernel\src\mem.o [2K
[53/56] Compiling build\kernel\src\scheduler.o [2K
[54/56] Compiling build\kernel\src\timer.o [2K
[55/56] Building rtthread-beaglebone.elf [2K
[56/56] Building rtthread.bin
scons: done building targets.
___________________________________________
------------------ 原始邮件 ------------------
发件人: "Daniel Moody" <dmoody256 at gmail.com>;
发送时间: 2022年5月18日(星期三) 上午6:11
收件人: "liruncong2018"<liruncong2018 at qq.com>;
抄送: "SCons users mailing list"<scons-users at scons.org>;
主题: Re: [Scons-users] 回复： env.Textfile problem
Also I added a new feature to my branch, now on the command line you can pass arguments to ninja invocation from scons command line with NINJA_CMD_ARGS=
So for example scons ... NINJA_CMD_ARGS="-n" will mean you will see ninja execute with the dry run arg '-n'.
On Tue, May 17, 2022 at 5:09 PM Daniel Moody <dmoody256 at gmail.com> wrote:
Hello liruncong,
I reproduced the problem. The problem is that scons does not update the ninja file if the existing ninja file looks exactly the same. Because the define option -DTEST2 is hidden a generated file, scons detects the ninja file is the same and does not update. Ninja also detects the the ninja file is the same, and does not call on scons to regenerate the defineOptsFile.
Originally you brought up the complaint that ninja always rebuilds some files and you think it should just say (ninja:no work to do). I added a new feature described previously to deal with this, but it seems it will not work with generated files like defineOptsFile. I reverted this feature in the ninja_integration_branch, you will have to see ninja calling to scons to check if some files should build, only scons can make the decision and ninja will detect if scons updated the files and if it should then rebuild things which depended on those files.
On Tue, May 17, 2022 at 7:02 AM liruncong2018 <liruncong2018 at qq.com> wrote:
Hello Daniel Moody,
Thanks for the reminder, I corrected the recompile problem (https://github.com/liruncong/NinJaTest/tree/EnvTextFile).
But the problem of not recompiling after modifying defineOptsFile, I still have no idea.
------------------ 原始邮件 ------------------
发件人: "Daniel Moody" <dmoody256 at gmail.com>;
发送时间: 2022年5月17日(星期二) 下午2:03
收件人: "SCons users mailing list"<scons-users at scons.org>;
抄送: "liruncong2018"<liruncong2018 at qq.com>;
主题: Re: [Scons-users] 回复： env.Textfile problem
hello liruncong,
First you should understand why it is recompiling every time with just scons. Are your @files being generated differently because of ordering of the contents? Can you pick one file which is always recompiled and use this as the target to build with --debug=explain option?
On Sun, May 15, 2022 at 4:20 AM liruncong2018 via Scons-users <scons-users at scons.org> wrote:
Hi,
I originally wanted to reproduce the problem in the test project, but I did find another problem, scons is recompiled every time.
Test code: liruncong/NinJaTest at EnvTextFile (github.com)
However, this problem was not found in my actual project.
The test project uses the gcc toolchain, and my actual project uses the armclang (clang) toolchain.
When ninja is not enabled, it is all recompiled every time you compile. After enabling ninja, the first two executions are all recompiles, and the third start is normal (ninja: no work to do.).
After normal (three times later), modify defineOptsFile as follows, scons (enable ninja) does not recompile, which should be problematic.
defineOptsFile = env.Textfile("build/__define_options.txt", [
"-DTEST1",
"-DTEST2" # new added
])
------------------ 原始邮件 ------------------
发件人: "Bill Deegan" <bill at baddogconsulting.com>;
发送时间: 2022年5月15日(星期天) 上午7:20
收件人: "SCons users mailing list"<scons-users at scons.org>;
抄送: "liruncong2018"<liruncong2018 at qq.com>;
主题: Re: [Scons-users] env.Textfile problem
Not enough information to help you.
What's your scons logic look like for this?
On Sat, May 14, 2022 at 3:04 AM liruncong2018 via Scons-users <scons-users at scons.org> wrote:
Hi,
The source(env.Textfile second parameter) change did not trigger recompilation when using env.Textfile.
scons version: Commits · dmoody256/scons (github.com) commit: force utf-8
_______________________________________________
Scons-users mailing list
Scons-users at scons.org
https://pairlist4.pair.net/mailman/listinfo/scons-users
_______________________________________________
Scons-users mailing list
Scons-users at scons.org
https://pairlist4.pair.net/mailman/listinfo/scons-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20220523/4c15c55c/attachment-0001.htm>
More information about the Scons-users
mailing list