[Scons-users] Fwd:Re: timing issues and protecting from them

[Tom Tanner (BLOOMBERG/ LONDON)]

Presuming this is all one build:

I think I'm correct in stating that the calculation of the md5 and timestamp of libabc_api.h is done once, before any of a.o, b.o, x.o are built.

So the (proposed) system would say
 Decide we need libabc_api.h, cache the timestamp and md5
 a.o - built, validates against timestamps
 b.o - built, validates against timestamps
 libabc_api.h changes
 libabc_abi.lib built, validates against .o timestamps
 x.o - built,
   libapc_api.h cached timestamp != libapc_api.h actual timestamp
   error


In passing, I'm poking around with the following implementation of Task.executed_with_callbacks

    def executed_with_callbacks(self):
        """
        Called when the task has been successfully executed and
        the Taskmaster instance wants to call the Node's callback
        methods.

        This may have been a do-nothing operation (to preserve build
        order), so we must check the node's state before deciding whether
        it was "built", in which case we call the appropriate Node method.
        In any event, we always call "visited()", which will handle any
        post-visit actions that must take place regardless of whether
        or not the target was an actual built target or a source Node.
        """
        T = self.tm.trace
        if T: T.write(self.trace_message('Task.executed_with_callbacks()',
                                         self.node))

        for t in self.targets:
            if t.get_state() == NODE_EXECUTING:
                for side_effect in t.side_effects:
                    side_effect.set_state(NODE_NO_STATE)
                t.set_state(NODE_EXECUTED)
                if not t.cached:
                    # ++++++++++++++++ ADDED CODE
                    for s in t.depends + t.executor.get_all_children():
                        if s.rexists():
                            ok = True
                            try:
                                s1 = s.get_timestamp()
                                s2 = s.rfile().getmtime()
                                if s1 != s2:
                                    ok = False
                            except:
                                pass
                            if not ok:
                                raise SCons.Errors.Builderror(s.str() + ' changed during build')
                    # ++++++++++++++++++++++++++++
                    t.cached = t.push_to_cache()
                t.built()
            t.visited()

executed_without_callback seems to be only used for clean, and I don't think this test would apply to clean!

Seems to be about enough. Not sure about the cost as it's only going to activate if you actually do a build without copying from cache, which I'd think is quite expensive in any case, so I shall have to bite the bullet and try a full build without using the cache.

From: bill at baddogconsulting.com At: Dec 10 2015 14:11:57
To: Tom Tanner (BLOOMBERG/ LONDON), scons-users at scons.org
Subject: Re: [Scons-users] Fwd:Re: timing issues and protecting from them

Tom,

How about this sequence.

a.o, b.o built agains libabc_api.h
libabc_api.h is modified in a way that breaks.. changing an enum for example
libabc_abi.lib built
x.o compiles and uses libabc_api.h
Your compile finishes
Now your code my break in strange and wonderous ways..

-Bill

On Thu, Dec 10, 2015 at 8:42 AM, Tom Tanner (BLOOMBERG/ LONDON) <ttanner2 at bloomberg.net> wrote:

Thanks. I'll start poking around in those places.

And yes, an indirect dependency can safely change because it isn't part of your signature. Only the direct dependencies are.

Consider:

a,o depends on a.hh, a.cc
a.lib depends on a.o

scons starts build
scons builds a.o and updates tree
idiot^H^H^H^H^Huser changes a.hh
scons build a.lib

You don't have an inconsistency. You do have a requirement to rebuild, but nothing is inconsistent.


From: dl9obn at darc.de At: Dec 10 2015 10:39:05To: scons-users at scons.org
Subject: Re: [Scons-users] Fwd:Re: timing issues and protecting from them

Tom,

On 10.12.2015 09:13, Tom Tanner (BLOOMBERG/ LONDON) wrote:
> Well, I can check the timing for that. But I don't see a necessity for 
rechecking the indirect dependencies
>

so an indirect dependency may change during a build, but a direct source not? 
This doesn't really make sense to me...but I 
understand where you're coming from. You have experienced this phenomenon of 
"hard to track down" build errors several times now, 
and want to do something about it. So please, as I mentioned in my earlier 
mail, start on an implementation for this feature if you 
find the time. The execute() and executed_with_callbacks() methods in the 
Taskmaster look like the best place to do that.
To me, this seems more like a job for the Taskmaster than putting new code in 
the Node() methods build()/built().
I'd derive a new ParanoidTaskmaster from the original class and then add my 
checks to that. This also includes generalizing the 
creation of the "Taskmaster" in the Main.py script, such that you can select 
between the "default" and your new version via a 
command-line switch.

Finally, the really paranoid people clearly separate builds from their edit 
cycles by building the final stuff on a different server 
for example. I happen to work in one of those places ;)...and we always build 
against fixed labels.

Best regards,

Dirk

_______________________________________________
Scons-users mailing list
Scons-users at scons.org
https://pairlist4.pair.net/mailman/listinfo/scons-users


_______________________________________________
Scons-users mailing list
Scons-users at scons.org
https://pairlist4.pair.net/mailman/listinfo/scons-users


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20151210/0b837f0f/attachment-0001.html>