[Scons-users] Scons none-deterministic behavior for incremental build
Hua Yanghao
huayanghao at gmail.com
Wed Dec 5 04:50:15 EST 2018
I finally nailed it down.
It is because the ordering of SConscript files processing, where in
SConscript I allow users/modules to provide additional CFLAGS.
I used a set() to eliminate duplicated SConscript, but in Python3
set() does not have a stable run-time ordering.
The fix is very simple to convert the set() back to list and sort it out.
So scons is behaving all correctly and it is my user code problem.
Thanks Oscar and Bill for all the hints!
Best Regards,
Yanghao
On Wed, Dec 5, 2018 at 9:38 AM Hua Yanghao <huayanghao at gmail.com> wrote:
>
> Alright, I have figured out a way to bypass/stub most of the python3
> code and can confirm this *ONLY* happens in python3. And python3.6.x
> and python3.7.x doesn't make a difference.
>
> But I think the problem is clearly that the previously stored md5sum
> and currently computed md5sum is not reliable, seems scons computation
> of md5sum of a dependency does not only depends on the file content.
> On Wed, Dec 5, 2018 at 9:23 AM Hua Yanghao <huayanghao at gmail.com> wrote:
> >
> > It is impossible to run on python2.7 within a reasonable time as a
> > huge list of python3-only syntax/modules needs to be adapted ...
> > On Wed, Dec 5, 2018 at 12:15 AM Bill Deegan <bill at baddogconsulting.com> wrote:
> > >
> > > Does this happen if you use python 2.7?
> > >
> > > On Tue, Dec 4, 2018 at 6:00 PM Hua Yanghao <huayanghao at gmail.com> wrote:
> > >>
> > >> Apparently the MD5 signature is taking *turns* some how.
> > >>
> > >> You can see below the old md5 and new md5 checksum is somehow
> > >> alternating between two fixed value ...
> > >> How is scons calculating the md5 value? is it only depends on the file content?
> > >> The md5sum of the "arch_init.c" is:
> > >> md5sum firmware/arch/armv7/arch_init.c
> > >> db624fbb365812d7ba58c87d49f44b6f firmware/arch/armv7/arch_init.c
> > >>
> > >> First build:
> > >> e494890ea1b6e36acf275f3698a3f3f9
> > >> c706a016770a7c85ac42f16f79456797
> > >> Returning True
> > >> in self.changed ... 2
> > >> return not r: False
> > >> ['build/configs/qemu_arm_vexpress/arm3/firmware/arch/armv7/arch_init.o']
> > >> <SCons.Script.Main.BuildTask object at 0x7fdc934c7fd0>
> > >> ______CC build/configs/qemu_arm_vexpress/arm3/firmware/arch/armv7/arch_init.o
> > >>
> > >> Second build:
> > >> c706a016770a7c85ac42f16f79456797
> > >> e494890ea1b6e36acf275f3698a3f3f9
> > >> Returning True
> > >> in self.changed ... 2
> > >> return not r: False
> > >> ['build/configs/qemu_arm_vexpress/arm3/firmware/arch/armv7/arch_init.o']
> > >> <SCons.Script.Main.BuildTask object at 0x7f3fca5e4fd0>
> > >> ______CC build/configs/qemu_arm_vexpress/arm3/firmware/arch/armv7/arch_init.o
> > >> On Tue, Dec 4, 2018 at 11:54 PM Hua Yanghao <huayanghao at gmail.com> wrote:
> > >> >
> > >> > That is generated by sconstruct, not from shell. Any potential issue
> > >> > here? (It was working for a very long time before and only breaks
> > >> > recently when I added some new features).
> > >> >
> > >> > In the meantime I have isolated the issue to be in Node.Node.changed()
> > >> > call, where scons old & new signature for the C files seems not
> > >> > matching (print("###### 3.1") is executed in error case):
> > >> > 1463 if self.has_builder():
> > >> > |3
> > >> > 1464 print("####### 3")
> > >> > |3
> > >> > 1465 import SCons.Util
> > >> > |3
> > >> > 1466 newsig = SCons.Util.MD5signature(contents)
> > >> > |3
> > >> > 1467 if bi.bactsig != newsig:
> > >> > |3
> > >> > 1468 if t: Trace(': bactsig %s != newsig %s' %
> > >> > (bi.bactsig, newsig)) |3
> > >> > 1469 print("####### 3.1")
> > >> > |3
> > >> > 1470 result = True
> > >> > On Tue, Dec 4, 2018 at 11:24 PM Bill Deegan <bill at baddogconsulting.com> wrote:
> > >> > >
> > >> > > Generated by logic in your sconstruct/sconscripts? or do you shell out and run a script/program to generate it.
> > >> > >
> > >> > >
> > >> > > On Tue, Dec 4, 2018 at 4:37 PM Hua Yanghao <huayanghao at gmail.com> wrote:
> > >> > >>
> > >> > >> Hi Bill,
> > >> > >> config.h is generated file, not checked in, it is generated in the
> > >> > >> build folder only.
> > >> > >> Well at least I see scons is very odering-sensitive. For example, if
> > >> > >> the order of SConscript processing changes (in the case of a set()),
> > >> > >> even if there is no dependency actually changed, scons thinks the
> > >> > >> target need to be rebuilt.
> > >> > >>
> > >> > >> any progress on these similar issues already? :)
> > >> > >>
> > >> > >> Best Regards,
> > >> > >> Yanghao
> > >> > >> On Tue, Dec 4, 2018 at 10:15 PM Bill Deegan <bill at baddogconsulting.com> wrote:
> > >> > >> >
> > >> > >> > Is "build/configs/qemu_arm_vexpress/arm3/config/config.h" a generated file? Or checked in?
> > >> > >> >
> > >> > >> > It's very unlikely to be set ordering. set's should only be used for membership checks not iteration in the SCons codebase.
> > >> > >> >
> > >> > >> > We're actively debugging similar issue.
> > >> > >> >
> > >> > >> > -Bill
> > >> > >> >
> > >> > >> > On Tue, Dec 4, 2018 at 3:23 PM Hua Yanghao <huayanghao at gmail.com> wrote:
> > >> > >> >>
> > >> > >> >> I does that a lot ... but a quick try to make sure no
> > >> > >> >> LIBS/LIBPATH/CPPPATH etc. is touched after Object() is being called,
> > >> > >> >> still shows the same 50~70% rebuild of the entire project.
> > >> > >> >> I am not using any parallel build, and somehow scons is having a very
> > >> > >> >> none-deterministic behavior.
> > >> > >> >>
> > >> > >> >> Actually I fixed partially the issue (e.g. in simpler build the
> > >> > >> >> incremental builds now works fine), due to python set() is having a
> > >> > >> >> random order for every run. I am thinking maybe scons is somehow also
> > >> > >> >> depends on the ordering of a set()? And when it happens to be the same
> > >> > >> >> order then no rebuild?
> > >> > >> >> On Tue, Dec 4, 2018 at 9:00 PM Oscar Anson <oscar at solidangle.com> wrote:
> > >> > >> >> >
> > >> > >> >> > Hi,
> > >> > >> >> >
> > >> > >> >> > Not sure if this could be related, but a similar issue happened to me where I get a relink of a shared library without changing anything in the project. It happened that I was setting the environment LIBS and LIBPATH after calling the builder of the library.
> > >> > >> >> >
> > >> > >> >> > El mar., 4 dic. 2018 20:55, Hua Yanghao <huayanghao at gmail.com> escribió:
> > >> > >> >> >>
> > >> > >> >> >> Dear Scons Experts,
> > >> > >> >> >> I recently hit on a very strange problem where scons incremental
> > >> > >> >> >> builds sometimes OK, sometimes the entire project is re-built.
> > >> > >> >> >>
> > >> > >> >> >> scons: 3.0.1
> > >> > >> >> >> python: 3.6.7
> > >> > >> >> >>
> > >> > >> >> >> --debug=explain gives below example trace:
> > >> > >> >> >> scons: rebuilding
> > >> > >> >> >> `build/configs/qemu_arm_vexpress/arm3/config/config.h' because
> > >> > >> >> >> AlwaysBuild() is specified
> > >> > >> >> >> __CONFIG build/configs/qemu_arm_vexpress/arm3/config/config.h
> > >> > >> >> >> scons: rebuilding
> > >> > >> >> >> `build/configs/qemu_arm_vexpress/arm3/firmware/arch/armv7/arch_init.o'
> > >> > >> >> >> because:
> > >> > >> >> >> `firmware/arch/armv7/arch_init.c' is no longer a dependency
> > >> > >> >> >> `firmware/arch/armv7/arch_init.c' is a new dependency
> > >> > >> >> >>
> > >> > >> >> >> It seems some how the C file itself is removed and re-added as the
> > >> > >> >> >> Object() dependency, and causes rebuild. However this is *NOT*
> > >> > >> >> >> *always* the case. Sometime the rebuild doesn't happen.
> > >> > >> >> >>
> > >> > >> >> >> I failed to create a minimal example that can show this issue so far,
> > >> > >> >> >> but in the production environment it is about 50% full-rebuild and 50%
> > >> > >> >> >> incremental re-build.
> > >> > >> >> >>
> > >> > >> >> >> Any thoughts or hints?
> > >> > >> >> >>
> > >> > >> >> >> Thanks,
> > >> > >> >> >> Yanghao
> > >> > >> >> >> _______________________________________________
> > >> > >> >> >> Scons-users mailing list
> > >> > >> >> >> Scons-users at scons.org
> > >> > >> >> >> https://pairlist4.pair.net/mailman/listinfo/scons-users
> > >> > >> >> >
> > >> > >> >> > _______________________________________________
> > >> > >> >> > Scons-users mailing list
> > >> > >> >> > Scons-users at scons.org
> > >> > >> >> > https://pairlist4.pair.net/mailman/listinfo/scons-users
> > >> > >> >> _______________________________________________
> > >> > >> >> Scons-users mailing list
> > >> > >> >> Scons-users at scons.org
> > >> > >> >> https://pairlist4.pair.net/mailman/listinfo/scons-users
> > >> > >> >
> > >> > >> > _______________________________________________
> > >> > >> > Scons-users mailing list
> > >> > >> > Scons-users at scons.org
> > >> > >> > https://pairlist4.pair.net/mailman/listinfo/scons-users
> > >> > >> _______________________________________________
> > >> > >> Scons-users mailing list
> > >> > >> Scons-users at scons.org
> > >> > >> https://pairlist4.pair.net/mailman/listinfo/scons-users
> > >> > >
> > >> > > _______________________________________________
> > >> > > Scons-users mailing list
> > >> > > Scons-users at scons.org
> > >> > > https://pairlist4.pair.net/mailman/listinfo/scons-users
> > >> _______________________________________________
> > >> Scons-users mailing list
> > >> Scons-users at scons.org
> > >> https://pairlist4.pair.net/mailman/listinfo/scons-users
> > >
> > > _______________________________________________
> > > Scons-users mailing list
> > > Scons-users at scons.org
> > > https://pairlist4.pair.net/mailman/listinfo/scons-users
More information about the Scons-users
mailing list