[Scons-users] Detecting changed targets

[Mark Terrel]

Hi SCons gurus!

I've been a user of SCons since 2010 (at a previous company) and am using
it for a new project at a new company now.  We're doing all builds and
development in containers, so I've created a SCons builder to build Docker
images.  (I'm happy to share that once it's had more testing.)  That's
working great so far, except for one issue that I'd like to solve.

The issue stems from the fact that Docker images are not stored in the
filesystem; they're stored in a common object store repository and are
tagged with a name.  So while a typical user of SCons has a reasonable
expectation that no one is going to come modify their build output
directory contents, it now becomes much more likely that Developer A builds
a Docker image with tag "production_container" and Developer B could build
a different image and overwrite (actually move the tag)
"production_container" to point to a different Docker image.  This makes it
harder to ensure correctness of builds because SCons does not currently
rebuild a target if the signature of the target changes.

I found this exchange
<http://scons-users.scons.narkive.com/7BJnwEox/corrupted-objects-are-not-rebuilt-as-one-might-expect>
on the scons-users list from ages ago, where Steven Knight suggested a new
ValidateTargets option that would detect if a target was changed (or had
been corrupted) unbeknownst to SCons and would then rebuild that target.
So I took a shot at implementing that feature and would love some feedback
on whether this is a good approach and whether, after it meets the dev
team's approval, you might be willing to incorporate it.

This commit
<https://bitbucket.org/mterrel/scons/commits/1b0c0f22d13596f709b6e3ca64a8f2ef41fd4c11>contains
the proposed feature as a proof of concept.  It has two new user-facing
functions:

*ValidateTargets('none' | 'content')*

Sets the global default behavior for all target nodes.
'none' is the default and is the same as SCons' current behavior, which is
to not check the signatures for nodes that are only targets.

'content' uses the existing content signatures to check whether a target
node has been modified outside of SCons' knowledge.

*env.ValidateTarget(target_list, 'none' | 'content')*

Sets the validation behavior for specific node(s), overriding the global
default behavior for those nodes.  Options 'none' and 'content' behave the
same as above.


​I think this new behavior contributes toward one of SCons' goals of
correctness of builds.  I think it would also be particularly useful for
other kinds of targets that might be in shared storage, such as AWS S3 or
other object storage.  Those types of output storage are becoming more and
more common as developers start distributing and testing builds across
cloud environments.

​So please take a look at the proposal and give me any feedback or ideas
and let me know if this is something I should try to move forward on.

Thanks!
Mark

P.S.  I tested this on top of 2.5.1 and am running regression tests now on
tip.

-- 
Mark Terrel  |  *Unbounded Systems*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20161228/907a052d/attachment-0001.html>