[Scons-users] Scons none-deterministic behavior for incremental build
Bill Deegan
bill at baddogconsulting.com
Tue Dec 4 18:07:41 EST 2018
Does this happen if you use python 2.7?
On Tue, Dec 4, 2018 at 6:00 PM Hua Yanghao <huayanghao at gmail.com> wrote:
> Apparently the MD5 signature is taking *turns* some how.
>
> You can see below the old md5 and new md5 checksum is somehow
> alternating between two fixed value ...
> How is scons calculating the md5 value? is it only depends on the file
> content?
> The md5sum of the "arch_init.c" is:
> md5sum firmware/arch/armv7/arch_init.c
> db624fbb365812d7ba58c87d49f44b6f firmware/arch/armv7/arch_init.c
>
> First build:
> e494890ea1b6e36acf275f3698a3f3f9
> c706a016770a7c85ac42f16f79456797
> Returning True
> in self.changed ... 2
> return not r: False
> ['build/configs/qemu_arm_vexpress/arm3/firmware/arch/armv7/arch_init.o']
> <SCons.Script.Main.BuildTask object at 0x7fdc934c7fd0>
> ______CC
> build/configs/qemu_arm_vexpress/arm3/firmware/arch/armv7/arch_init.o
>
> Second build:
> c706a016770a7c85ac42f16f79456797
> e494890ea1b6e36acf275f3698a3f3f9
> Returning True
> in self.changed ... 2
> return not r: False
> ['build/configs/qemu_arm_vexpress/arm3/firmware/arch/armv7/arch_init.o']
> <SCons.Script.Main.BuildTask object at 0x7f3fca5e4fd0>
> ______CC
> build/configs/qemu_arm_vexpress/arm3/firmware/arch/armv7/arch_init.o
> On Tue, Dec 4, 2018 at 11:54 PM Hua Yanghao <huayanghao at gmail.com> wrote:
> >
> > That is generated by sconstruct, not from shell. Any potential issue
> > here? (It was working for a very long time before and only breaks
> > recently when I added some new features).
> >
> > In the meantime I have isolated the issue to be in Node.Node.changed()
> > call, where scons old & new signature for the C files seems not
> > matching (print("###### 3.1") is executed in error case):
> > 1463 if self.has_builder():
> > |3
> > 1464 print("####### 3")
> > |3
> > 1465 import SCons.Util
> > |3
> > 1466 newsig = SCons.Util.MD5signature(contents)
> > |3
> > 1467 if bi.bactsig != newsig:
> > |3
> > 1468 if t: Trace(': bactsig %s != newsig %s' %
> > (bi.bactsig, newsig)) |3
> > 1469 print("####### 3.1")
> > |3
> > 1470 result = True
> > On Tue, Dec 4, 2018 at 11:24 PM Bill Deegan <bill at baddogconsulting.com>
> wrote:
> > >
> > > Generated by logic in your sconstruct/sconscripts? or do you shell out
> and run a script/program to generate it.
> > >
> > >
> > > On Tue, Dec 4, 2018 at 4:37 PM Hua Yanghao <huayanghao at gmail.com>
> wrote:
> > >>
> > >> Hi Bill,
> > >> config.h is generated file, not checked in, it is generated in the
> > >> build folder only.
> > >> Well at least I see scons is very odering-sensitive. For example, if
> > >> the order of SConscript processing changes (in the case of a set()),
> > >> even if there is no dependency actually changed, scons thinks the
> > >> target need to be rebuilt.
> > >>
> > >> any progress on these similar issues already? :)
> > >>
> > >> Best Regards,
> > >> Yanghao
> > >> On Tue, Dec 4, 2018 at 10:15 PM Bill Deegan <
> bill at baddogconsulting.com> wrote:
> > >> >
> > >> > Is "build/configs/qemu_arm_vexpress/arm3/config/config.h" a
> generated file? Or checked in?
> > >> >
> > >> > It's very unlikely to be set ordering. set's should only be used
> for membership checks not iteration in the SCons codebase.
> > >> >
> > >> > We're actively debugging similar issue.
> > >> >
> > >> > -Bill
> > >> >
> > >> > On Tue, Dec 4, 2018 at 3:23 PM Hua Yanghao <huayanghao at gmail.com>
> wrote:
> > >> >>
> > >> >> I does that a lot ... but a quick try to make sure no
> > >> >> LIBS/LIBPATH/CPPPATH etc. is touched after Object() is being
> called,
> > >> >> still shows the same 50~70% rebuild of the entire project.
> > >> >> I am not using any parallel build, and somehow scons is having a
> very
> > >> >> none-deterministic behavior.
> > >> >>
> > >> >> Actually I fixed partially the issue (e.g. in simpler build the
> > >> >> incremental builds now works fine), due to python set() is having a
> > >> >> random order for every run. I am thinking maybe scons is somehow
> also
> > >> >> depends on the ordering of a set()? And when it happens to be the
> same
> > >> >> order then no rebuild?
> > >> >> On Tue, Dec 4, 2018 at 9:00 PM Oscar Anson <oscar at solidangle.com>
> wrote:
> > >> >> >
> > >> >> > Hi,
> > >> >> >
> > >> >> > Not sure if this could be related, but a similar issue happened
> to me where I get a relink of a shared library without changing anything in
> the project. It happened that I was setting the environment LIBS and
> LIBPATH after calling the builder of the library.
> > >> >> >
> > >> >> > El mar., 4 dic. 2018 20:55, Hua Yanghao <huayanghao at gmail.com>
> escribió:
> > >> >> >>
> > >> >> >> Dear Scons Experts,
> > >> >> >> I recently hit on a very strange problem where scons incremental
> > >> >> >> builds sometimes OK, sometimes the entire project is re-built.
> > >> >> >>
> > >> >> >> scons: 3.0.1
> > >> >> >> python: 3.6.7
> > >> >> >>
> > >> >> >> --debug=explain gives below example trace:
> > >> >> >> scons: rebuilding
> > >> >> >> `build/configs/qemu_arm_vexpress/arm3/config/config.h' because
> > >> >> >> AlwaysBuild() is specified
> > >> >> >> __CONFIG build/configs/qemu_arm_vexpress/arm3/config/config.h
> > >> >> >> scons: rebuilding
> > >> >> >>
> `build/configs/qemu_arm_vexpress/arm3/firmware/arch/armv7/arch_init.o'
> > >> >> >> because:
> > >> >> >> `firmware/arch/armv7/arch_init.c' is no longer a
> dependency
> > >> >> >> `firmware/arch/armv7/arch_init.c' is a new dependency
> > >> >> >>
> > >> >> >> It seems some how the C file itself is removed and re-added as
> the
> > >> >> >> Object() dependency, and causes rebuild. However this is *NOT*
> > >> >> >> *always* the case. Sometime the rebuild doesn't happen.
> > >> >> >>
> > >> >> >> I failed to create a minimal example that can show this issue
> so far,
> > >> >> >> but in the production environment it is about 50% full-rebuild
> and 50%
> > >> >> >> incremental re-build.
> > >> >> >>
> > >> >> >> Any thoughts or hints?
> > >> >> >>
> > >> >> >> Thanks,
> > >> >> >> Yanghao
> > >> >> >> _______________________________________________
> > >> >> >> Scons-users mailing list
> > >> >> >> Scons-users at scons.org
> > >> >> >> https://pairlist4.pair.net/mailman/listinfo/scons-users
> > >> >> >
> > >> >> > _______________________________________________
> > >> >> > Scons-users mailing list
> > >> >> > Scons-users at scons.org
> > >> >> > https://pairlist4.pair.net/mailman/listinfo/scons-users
> > >> >> _______________________________________________
> > >> >> Scons-users mailing list
> > >> >> Scons-users at scons.org
> > >> >> https://pairlist4.pair.net/mailman/listinfo/scons-users
> > >> >
> > >> > _______________________________________________
> > >> > Scons-users mailing list
> > >> > Scons-users at scons.org
> > >> > https://pairlist4.pair.net/mailman/listinfo/scons-users
> > >> _______________________________________________
> > >> Scons-users mailing list
> > >> Scons-users at scons.org
> > >> https://pairlist4.pair.net/mailman/listinfo/scons-users
> > >
> > > _______________________________________________
> > > Scons-users mailing list
> > > Scons-users at scons.org
> > > https://pairlist4.pair.net/mailman/listinfo/scons-users
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20181204/99a69c17/attachment-0001.html>
More information about the Scons-users
mailing list