[Scons-users] AddPostAction memoization problem

[Mats Wichmann]

On 5/29/24 02:18, Mike Haboustak wrote:
> Bill,
> 
> Our real-life build defines a Tool: EnsureMaxSize(target, max_size).
> We use it to enforce a size limit policy on a build output. The
> Builder for the target fails if the produced file exceeds the limit.
> The Tool calls AddPostAction on the target with an ActionFunction that
> renames the target (target + ".toobig") and raises a StopError if the
> value returned by target.get_size() exceeds the max_size argument.
> 
> MAX_APP_SIZE = 8 * 1024 * 1024
> 
> env = Environment()
> env.Tool("ensure_max_size")
> app = env.Program(target="myapp", source="main.c")
> env.EnsureMaxSize(app, MAX_APP_SIZE)
> 
> We don't use a Builder for a Tool like this, because we don't want to
> introduce a new target / Node / file. For example, we could write:
> env.EnsureMaxSize(target=app+".sizeok", source=app), but we don't want
> any further nodes in the dependency graph that depend on "app" to be
> built using the too-large file, and we don't want to add app.sizeok to
> the dependency graph in place of app.
> 
> By not introducing a new Node, we may be making some unsafe
> assumptions about the Actions within the Builder.
> 1. Our use of PostAction assumes the primary Actions for building a
> target have finished.

Actions are executed in category order as Bill said

     for act in obj.get_action_list():
         ...
         status = act(*args, **kw)
         # if status doesn't indicate success, bail 


where get_action_list had done:

     return self.pre_actions + self.action_list + self.post_actions

So build actions will be done by the time you get to any post actions.

Pre actions are a little different because the build actions in 
self.action_list may be just the end of a chain of steps (the manpage 
goes into this, I believe).

> 2. The execution order of PostActions can be significant if you were
> to combine Tools with operations like env.Compress(app) and
> env.EnsureMaxSize(app).

There's no way to specify the ordering other than "insertion order", 
each call to AddPostAction just appends to a list.
> Generally though, our Builders are simple, and the code we use to
> build a thing is close to the tools we invoke to post-process the
> thing. We're unlikely to write a Tool like env.Compress that would
> modify the content of a target in a PostAction. For us, a tool like
> that would be written as a Builder and the compressed file would be
> part of the dependency graph.
> 
> As to the correct behavior, every Action in a Builder's Action list
> has the opportunity to modify a target and or a side effect. I think
> we should call a function similar to invalidate_node_memos after each
> Action executes. The set of invalidated nodes should be the union of
> the Action's target list and each target's side_effect list. I'm not
> sure how much performance would be lost by clearing the cache for
> targets and side effects.

I'd personally agree as to correct behavior: if a step that makes 
changes has completed, information related to the node should be up to 
date and not old.  I guess it's worth mentioning that the getsize() 
method isn't *officially* exposed as a public interface, it's possible 
there was an actual reason for that back in the mists of time.

It should be sufficient to just remove the memoized 'stat', since it's 
the thing that holds information SCons itself doesn't update - it 
eventually comes from an os.stat() call - that could be considered 
"expensive" and thus worth memoizing. Side effects are, of course, an 
extra complication. However, the way things are encapsulated, you can't 
do that cleanly: the actions list belongs to the Executor, and it's not 
supposed to reach in and fiddle a Node's "private" attributes directly 
(the memo dict is such), and there's not a method for that (perhaps 
there should be?).  There are, however, already places that do this so 
it's not a a completely unbreakable rule.

> I don't know that there's a practical way for an Action to express the
> set of targets or side effects that it has invalidated.

Stepping aside from the internal implementation questions, you could in 
this particular case do something that sidesteps the memoized stat 
structure, like:

     print("PostAction:", os.path.getsize(target[0].abspath))