[Scons-users] 回复： env.Textfile problem
Daniel Moody
dmoody256 at gmail.com
Mon May 23 10:46:32 EDT 2022
Hello Liruncong,
> But in the future, the difficulty for users should be reduced, and it is
difficult for users to solve these problems by themselves!
Which difficulties? The ninja tool is a new feature we are actively working
on improvements. I thank you for your time testing the tool and useful
debugging feedback.
> echo_response_file will cause the command line command to be too long!
If instead of '&' character in the echo_response_file function, you use a
'\n', will the command line be too long?
> Can env.Text be improved?
Some of the builtin SCons builder tools have not been translated to ninja
commands yet. In the future, scons can automatically create a command line
which represents env.Textfile call which ninja can run without needing
scons, but we have not translated this to ninja commands yet, so by default
ninja will callback to scons to do this for now.
> Let the txt file be regenerated when the string changes? Calling
run_ninja_env.bat does not regenerate it, I understand this. But using
"scons --experimental=ninja ..." did not regenerate the txt file, is this
just a bug?
This is not a bug. When scons is in ninja generation mode, it is only
creating the ninja file. Ninja must manage the build order, and the order
will matter, so not always can things be generated during the ninja
generation step. Therefore ninja must execute the steps during the ninja
build, and ninja will callback to scons to perform steps ninja can not do
it self, however ninja will manage the order of the steps.
On Mon, May 23, 2022 at 3:16 AM liruncong <liruncong2018 at qq.com> wrote:
> Hello Daniel Moody,
> Can env.Text be improved? Let the txt file be regenerated when the string
> changes? Calling run_ninja_env.bat does not regenerate it, I understand
> this. But using "scons --experimental=ninja ..." did not regenerate the txt
> file, is this just a bug?
>
> ------------------ 原始邮件 ------------------
> *发件人:* "liruncong" <liruncong2018 at qq.com>;
> *发送时间:* 2022年5月23日(星期一) 下午3:01
> *收件人:* "Daniel Moody"<dmoody256 at gmail.com>;
> *抄送:* "scons-users"<scons-users at scons.org>;
> *主题:* 回复： [Scons-users] 回复： env.Textfile problem
>
> Hello Daniel Moody,
> echo_response_file will cause the command line command to be too long!
> __cpp_path.txt is used to solve the problem of too long commands, but now
> after switching to env.Command, it is put on the command line again.
> __________________________________________
> def echo_response_file(respone_list):
> return (f'@echo {respone_list[0]} > $TARGET & ' +
> ' '.join(f'echo {item} >> $TARGET &' for item in
> respone_list[1:-1]) +
> f' echo {respone_list[-1]} >> $TARGET')
>
> compileIncPath = os.path.normpath(os.path.join(rtconfig.EXEC_PATH, "..",
> "include"))
> cppPathStr = [
> "-I%s" % (os.path.join(compileIncPath, "libcxx").replace("\\", "/")),
> "-I%s" % (compileIncPath.replace("\\", "/")),
> ]
> cppPathStr += ['-I' + path.replace("\\", "/") for path in env["CPPPATH"]]
> incPathsFile = env.Command("build/__cpp_path.txt",
> [],
> echo_response_file(cppPathStr)
> )
> __________________________________________
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "liruncong" <liruncong2018 at qq.com>;
> *发送时间:* 2022年5月23日(星期一) 下午2:47
> *收件人:* "Daniel Moody"<dmoody256 at gmail.com>;
> *抄送:* "scons-users"<scons-users at scons.org>;
> *主题:* 回复： [Scons-users] 回复： env.Textfile problem
>
> Hello Daniel Moody,
> Thank you!
> But in the future, the difficulty for users should be reduced, and it is
> difficult for users to solve these problems by themselves!
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "Daniel Moody" <dmoody256 at gmail.com>;
> *发送时间:* 2022年5月23日(星期一) 中午1:53
> *收件人:* "liruncong"<liruncong2018 at qq.com>;
> *抄送:* "scons-users"<scons-users at scons.org>;
> *主题:* Re: [Scons-users] 回复： env.Textfile problem
>
> Hello liruncong,
>
> For issue number 1:
> with scons only, I ran the build with --debug=explain and the output
> stated the order of the object files is changing every build. I fixed it
> with a sorted list like this:
> @@ -816,7 +816,7 @@ def DoBuilding(target, objects):
> for group in Projects:
> local_group(group, objects)
>
> - program = Env.Program(target, objects)
> + program = Env.Program(target, sorted(objects))
>
> EndBuilding(target, program)
>
> For issue number 2:
> this is unavoidable. As I explained, ninja can not know if those objects
> need to be rebuilt because only scons can know this, so it must call to
> scons to check. This is the output you will see, ninja is asking scons if
> these files need to be built again because of some scons change. Ninja can
> not know this, so it must always ask. SCons will see that the files do not
> need to change and it should not touch the files if they are already built.
> Ninja will see after it is done asking scons that the files have not been
> touched also and it will know not to rebuild any files which depended on
> those files which it asked scons to build.
>
> Alternatively, you do not need scons to build these files directly. If you
> convert the creation of the files to a command that ninja can execute on
> the command line then ninja will not need to callback to scons. I showed an
> example in the PR: https://github.com/liruncong/NinJaTest/pull/2
>
> For issue number 3:
> This is expected behavior, ninja "-n" option is dry-run which means it
> won't build anything, just test graph execution. I was using "-n" as an
> example of passing a ninja argument to ninja from scons.
>
>
> On Wed, May 18, 2022 at 7:01 AM liruncong2018 <liruncong2018 at qq.com>
> wrote:
>
>> Hi,
>> After updated,
>> 1) When ninja is not enabled, the last link is always executed when the
>> compilation is repeated.
>> log:
>> ___________________________________________
>> scons: Reading SConscript files ...
>> Newlib version:2.4.0
>> scons: done reading SConscript files.
>> scons: Building targets ...
>> scons: building associated VariantDir targets: build
>> arm-none-eabi-gcc -o rtthread-beaglebone.elf -Wall -march=armv7-a
>> -mtune=cortex-a8 -ftree-vectorize -ffast-math -mfpu=vfpv3-d16
>> -mfloat-abi=softfp
>> -Wl,--gc-sections,-Map=rtthread-beaglebone.map,-cref,-u,Reset_Handler -T
>> beaglebone_ram.lds -lm -lc @build/__link_objects.txt
>> scons: done building targets.
>> ___________________________________________
>> 2) When ninja is enabled, when recompiling, it always prints something
>> that doesn't seem to actually execute the steps
>> log:
>> ___________________________________________
>> [1/56] Starting scons daemon...[2K
>> [2/56] Defer to SCons to build build\__cpp_path.txt[2K
>> [3/56] Defer to SCons to build build\__define_options.txt[2K
>> [4/56] Defer to SCons to build build\__link_objects.txt
>> scons: done building targets.
>> ___________________________________________
>> Judging from the file time, these txt
>> files(__cpp_path.txt/__define_options.txt/__link_objects.txt) are not
>> actually generated.
>>
>> The following is the log using NINJA_CMD_ARGS, printing the same message
>> every time:
>> ___________________________________________
>> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>scons
>> -j%NUMBER_OF_PROCESSORS% --implicit-cache NINJA_CMD_ARGS="-n" --verbose
>> --experimental=ninja
>> scons: Reading SConscript files ...
>> Newlib version:2.4.0
>> scons: done reading SConscript files.
>> scons: Building targets ...
>> Generating: build.ninja...
>> No updates detected, build.ninja not regenerated.
>> Executing: run_ninja_env.bat -n
>> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set SystemDrive=C:
>> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set SystemRoot=C:\Windows
>> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set
>> TEMP=C:\Users\ARCHYI~1\AppData\Local\Temp
>> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set
>> TMP=C:\Users\ARCHYI~1\AppData\Local\Temp
>> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set
>> USERPROFILE=C:\Users\archyizimi
>> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set
>> COMSPEC=C:\Windows\system32\cmd.exe
>> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set
>> PATH=D:\MYBIN\env\tools\gnu_gcc\arm_gcc\mingw\bin;D:\MYBIN\scons-local-4.3.1\;D:\MYBIN;D:\MYBIN\Anaconda3\;C:\Program
>> Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\Llvm\bin;C:\Program
>> Files\Microsoft Visual
>> Studio\2022\Enterprise\VC\Tools\MSVC\14.32.31326\bin\HostX86\x86;C:\Program
>> Files\Microsoft Visual
>> Studio\2022\Enterprise\Common7\IDE\VC\VCPackages;C:\Program Files\Microsoft
>> Visual
>> Studio\2022\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow;C:\Program
>> Files\Microsoft Visual
>> Studio\2022\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team
>> Explorer;C:\Program Files\Microsoft Visual
>> Studio\2022\Enterprise\MSBuild\Current\bin\Roslyn;C:\Program
>> Files\Microsoft Visual Studio\2022\Enterprise\Team Tools\Performance
>> Tools;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8
>> Tools\;C:\Program Files (x86)\Windows
>> Kits\10\bin\10.0.19041.0\\x86;C:\Program Files (x86)\Windows
>> Kits\10\bin\\x86;C:\Program Files\Microsoft Visual
>> Studio\2022\Enterprise\\MSBuild\Current\Bin\amd64;C:\Windows\Microsoft.NET\Framework\v4.0.30319;C:\Program
>> Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\;C:\Program
>> Files\Microsoft Visual
>> Studio\2022\Enterprise\Common7\Tools\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program
>> Files\Microsoft SQL Server\150\Tools\Binn\;C:\Program Files\IDM Computer
>> Solutions\UltraEdit;C:\Program Files\IDM Computer
>> Solutions\UltraCompare;C:\Program Files\TortoiseGit\bin;C:\Program
>> Files\dotnet\;C:\Program
>> Files\Git\cmd;C:\Users\archyizimi\AppData\Local\Microsoft\WindowsApps;C:\Users\archyizimi\.dotnet\tools;C:\Program
>> Files\Microsoft Visual
>> Studio\2022\Enterprise\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin;C:\Program
>> Files\Microsoft Visual
>> Studio\2022\Enterprise\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja;C:\Program
>> Files\Microsoft Visual
>> Studio\2022\Enterprise\Common7\IDE\VC\Linux\bin\ConnectionManagerExe;c:\MinGW\bin
>> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>set
>> PATHEXT=.COM;.EXE;.BAT;.CMD
>> D:\work\gitee\rt-thread_ninja\bsp\beaglebone>D:\MYBIN\Anaconda3\lib\site-packages\ninja\data\bin\ninja.exe
>> -f D:\work\gitee\rt-thread_ninja\bsp\beaglebone\build.ninja -n
>> [1/56] Starting scons daemon... [2K
>> [2/56] Defer to SCons to build build\__cpp_path.txt [2K
>> [3/56] Defer to SCons to build build\__link_objects.txt [2K
>> [4/56] Defer to SCons to build build\__define_options.txt [2K
>> [5/56] Compiling build\applications\board.o [2K
>> [6/56] Compiling build\applications\main.o [2K
>> [7/56] Compiling build\drivers\gpio.o [2K
>> [8/56] Compiling build\kernel\components\dfs\src\dfs_posix.o [2K
>> [9/56] Compiling build\kernel\components\dfs\src\dfs_file.o [2K
>> [10/56] Compiling build\drivers\uart.o [2K
>> [11/56] Compiling build\kernel\components\dfs\filesystems\devfs\devfs.o
>> [2K
>> [12/56] Compiling build\kernel\components\dfs\src\dfs.o [2K
>> [13/56] Compiling build\kernel\components\dfs\src\dfs_fs.o [2K
>> [14/56] Compiling build\kernel\components\drivers\ipc\pipe.o [2K
>> [15/56] Compiling build\kernel\components\drivers\ipc\completion.o [2K
>> [16/56] Compiling build\kernel\components\drivers\ipc\dataqueue.o [2K
>> [17/56] Compiling build\kernel\components\drivers\ipc\ringbuffer.o [2K
>> [18/56] Compiling build\kernel\components\drivers\ipc\ringblk_buf.o [2K
>> [19/56] Compiling build\kernel\components\drivers\ipc\waitqueue.o [2K
>> [20/56] Compiling build\kernel\components\finsh\msh.o [2K
>> [21/56] Compiling build\kernel\components\drivers\serial\serial.o [2K
>> [22/56] Compiling build\kernel\components\drivers\ipc\workqueue.o [2K
>> [23/56] Compiling build\kernel\components\drivers\misc\pin.o [2K
>> [24/56] Compiling build\kernel\components\finsh\cmd.o [2K
>> [25/56] Compiling build\kernel\components\libc\compilers\common\time.o [2K
>> [26/56] Compiling build\kernel\components\finsh\shell.o [2K
>> [27/56] Compiling build\kernel\components\libc\compilers\common\stdlib.o
>> [2K
>> [28/56] Building build\kernel\libcpu\arm\am335x\context_gcc.o [2K
>> [29/56] Compiling
>> build\kernel\components\libc\compilers\newlib\syscalls.o [2K
>> [30/56] Compiling build\kernel\components\finsh\msh_file.o [2K
>> [31/56] Building build\kernel\libcpu\arm\am335x\start_gcc.o [2K
>> [32/56] Building build\kernel\libcpu\arm\am335x\cp15_gcc.o [2K
>> [33/56] Compiling build\kernel\libcpu\arm\am335x\interrupt.o [2K
>> [34/56] Compiling build\kernel\libcpu\arm\am335x\cpu.o [2K
>> [35/56] Compiling build\kernel\libcpu\arm\am335x\stack.o [2K
>> [36/56] Compiling build\kernel\libcpu\arm\am335x\mmu.o [2K
>> [37/56] Compiling build\kernel\libcpu\arm\am335x\trap.o [2K
>> [38/56] Compiling build\kernel\src\clock.o [2K
>> [39/56] Compiling build\kernel\libcpu\arm\common\backtrace.o [2K
>> [40/56] Compiling build\kernel\libcpu\arm\common\div0.o [2K
>> [41/56] Compiling build\kernel\libcpu\arm\common\showmem.o [2K
>> [42/56] Building build\kernel\libcpu\arm\am335x\vector_gcc.o [2K
>> [43/56] Compiling build\kernel\src\device.o [2K
>> [44/56] Compiling build\kernel\src\components.o [2K
>> [45/56] Compiling build\kernel\src\idle.o [2K
>> [46/56] Compiling build\kernel\src\irq.o [2K
>> [47/56] Compiling build\kernel\src\ipc.o [2K
>> [48/56] Compiling build\kernel\src\kservice.o [2K
>> [49/56] Compiling build\kernel\src\mempool.o [2K
>> [50/56] Compiling build\kernel\src\object.o [2K
>> [51/56] Compiling build\kernel\src\thread.o [2K
>> [52/56] Compiling build\kernel\src\mem.o [2K
>> [53/56] Compiling build\kernel\src\scheduler.o [2K
>> [54/56] Compiling build\kernel\src\timer.o [2K
>> [55/56] Building rtthread-beaglebone.elf [2K
>> [56/56] Building rtthread.bin
>> scons: done building targets.
>> ___________________________________________
>>
>>
>> ------------------ 原始邮件 ------------------
>> *发件人:* "Daniel Moody" <dmoody256 at gmail.com>;
>> *发送时间:* 2022年5月18日(星期三) 上午6:11
>> *收件人:* "liruncong2018"<liruncong2018 at qq.com>;
>> *抄送:* "SCons users mailing list"<scons-users at scons.org>;
>> *主题:* Re: [Scons-users] 回复： env.Textfile problem
>>
>> Also I added a new feature to my branch, now on the command line you can
>> pass arguments to ninja invocation from scons command line with
>> NINJA_CMD_ARGS=
>>
>> So for example scons ... NINJA_CMD_ARGS="-n" will mean you will see
>> ninja execute with the dry run arg '-n'.
>>
>> On Tue, May 17, 2022 at 5:09 PM Daniel Moody <dmoody256 at gmail.com> wrote:
>>
>>> Hello liruncong,
>>>
>>> I reproduced the problem. The problem is that scons does not update the
>>> ninja file if the existing ninja file looks exactly the same. Because the
>>> define option -DTEST2 is hidden a generated file, scons detects the ninja
>>> file is the same and does not update. Ninja also detects the the ninja file
>>> is the same, and does not call on scons to regenerate the defineOptsFile.
>>>
>>> Originally you brought up the complaint that ninja always rebuilds some
>>> files and you think it should just say (ninja:no work to do). I added a new
>>> feature described previously to deal with this, but it seems it will not
>>> work with generated files like defineOptsFile. I reverted this feature in
>>> the ninja_integration_branch, you will have to see ninja calling to scons
>>> to check if some files should build, only scons can make the decision and
>>> ninja will detect if scons updated the files and if it should then rebuild
>>> things which depended on those files.
>>>
>>> On Tue, May 17, 2022 at 7:02 AM liruncong2018 <liruncong2018 at qq.com>
>>> wrote:
>>>
>>>> Hello Daniel Moody,
>>>> Thanks for the reminder, I corrected the recompile problem (
>>>> https://github.com/liruncong/NinJaTest/tree/EnvTextFile).
>>>> But the problem of not recompiling after modifying defineOptsFile, I
>>>> still have no idea.
>>>>
>>>>
>>>> ------------------ 原始邮件 ------------------
>>>> *发件人:* "Daniel Moody" <dmoody256 at gmail.com>;
>>>> *发送时间:* 2022年5月17日(星期二) 下午2:03
>>>> *收件人:* "SCons users mailing list"<scons-users at scons.org>;
>>>> *抄送:* "liruncong2018"<liruncong2018 at qq.com>;
>>>> *主题:* Re: [Scons-users] 回复： env.Textfile problem
>>>>
>>>> hello liruncong,
>>>>
>>>> First you should understand why it is recompiling every time with just
>>>> scons. Are your @files being generated differently because of ordering of
>>>> the contents? Can you pick one file which is always recompiled and use this
>>>> as the target to build with --debug=explain option?
>>>>
>>>>
>>>>
>>>> On Sun, May 15, 2022 at 4:20 AM liruncong2018 via Scons-users <
>>>> scons-users at scons.org> wrote:
>>>>
>>>>> Hi,
>>>>> I originally wanted to reproduce the problem in the test project, but
>>>>> I did find another problem, scons is recompiled every time.
>>>>> Test code: liruncong/NinJaTest at EnvTextFile (github.com)
>>>>> <https://github.com/liruncong/NinJaTest/tree/EnvTextFile>
>>>>> However, this problem was not found in my actual project.
>>>>> The test project uses the gcc toolchain, and my actual project uses
>>>>> the armclang (clang) toolchain.
>>>>>
>>>>> When ninja is not enabled, it is all recompiled every time you
>>>>> compile. After enabling ninja, the first two executions are all recompiles,
>>>>> and the third start is normal (ninja: no work to do.).
>>>>> After normal (three times later), modify defineOptsFile as follows,
>>>>> scons (enable ninja) does not recompile, which should be problematic.
>>>>> defineOptsFile = env.Textfile("build/__define_options.txt", [
>>>>> "-DTEST1",
>>>>> "-DTEST2" # new added
>>>>> ])
>>>>>
>>>>> ------------------ 原始邮件 ------------------
>>>>> *发件人:* "Bill Deegan" <bill at baddogconsulting.com>;
>>>>> *发送时间:* 2022年5月15日(星期天) 上午7:20
>>>>> *收件人:* "SCons users mailing list"<scons-users at scons.org>;
>>>>> *抄送:* "liruncong2018"<liruncong2018 at qq.com>;
>>>>> *主题:* Re: [Scons-users] env.Textfile problem
>>>>>
>>>>> Not enough information to help you.
>>>>>
>>>>> What's your scons logic look like for this?
>>>>>
>>>>> On Sat, May 14, 2022 at 3:04 AM liruncong2018 via Scons-users <
>>>>> scons-users at scons.org> wrote:
>>>>>
>>>>>> Hi,
>>>>>> The source(env.Textfile second parameter) change did not trigger
>>>>>> recompilation when using env.Textfile.
>>>>>> scons version: Commits · dmoody256/scons (github.com)
>>>>>> <https://github.com/dmoody256/scons/commits/ninja_integration_branch>
>>>>>> commit: force utf-8
>>>>>> <https://github.com/dmoody256/scons/commit/c046aa853c177506543c0510562121b04f189e87>
>>>>>> _______________________________________________
>>>>>> Scons-users mailing list
>>>>>> Scons-users at scons.org
>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>
>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20220523/cf60d91c/attachment-0001.htm>
More information about the Scons-users
mailing list