[Scons-users] Detecting changed targets

[Mark Terrel]

Hi SCons folks,

Now that the holidays are over, I was hoping someone might give me feedback
on my proposal.  Any thoughts?

Thanks,
Mark

On Wed, Dec 28, 2016 at 3:01 PM, Mark Terrel <mark at unbounded.systems> wrote:

> Hi SCons gurus!
>
> I've been a user of SCons since 2010 (at a previous company) and am using
> it for a new project at a new company now.  We're doing all builds and
> development in containers, so I've created a SCons builder to build Docker
> images.  (I'm happy to share that once it's had more testing.)  That's
> working great so far, except for one issue that I'd like to solve.
>
> The issue stems from the fact that Docker images are not stored in the
> filesystem; they're stored in a common object store repository and are
> tagged with a name.  So while a typical user of SCons has a reasonable
> expectation that no one is going to come modify their build output
> directory contents, it now becomes much more likely that Developer A builds
> a Docker image with tag "production_container" and Developer B could build
> a different image and overwrite (actually move the tag)
> "production_container" to point to a different Docker image.  This makes it
> harder to ensure correctness of builds because SCons does not currently
> rebuild a target if the signature of the target changes.
>
> I found this exchange
> <http://scons-users.scons.narkive.com/7BJnwEox/corrupted-objects-are-not-rebuilt-as-one-might-expect>
> on the scons-users list from ages ago, where Steven Knight suggested a new
> ValidateTargets option that would detect if a target was changed (or had
> been corrupted) unbeknownst to SCons and would then rebuild that target.
> So I took a shot at implementing that feature and would love some feedback
> on whether this is a good approach and whether, after it meets the dev
> team's approval, you might be willing to incorporate it.
>
> This commit
> <https://bitbucket.org/mterrel/scons/commits/1b0c0f22d13596f709b6e3ca64a8f2ef41fd4c11>contains
> the proposed feature as a proof of concept.  It has two new user-facing
> functions:
>
> *ValidateTargets('none' | 'content')*
>
> Sets the global default behavior for all target nodes.
> 'none' is the default and is the same as SCons' current behavior, which is
> to not check the signatures for nodes that are only targets.
>
> 'content' uses the existing content signatures to check whether a target
> node has been modified outside of SCons' knowledge.
>
> *env.ValidateTarget(target_list, 'none' | 'content')*
>
> Sets the validation behavior for specific node(s), overriding the global
> default behavior for those nodes.  Options 'none' and 'content' behave the
> same as above.
>
>
> ​I think this new behavior contributes toward one of SCons' goals of
> correctness of builds.  I think it would also be particularly useful for
> other kinds of targets that might be in shared storage, such as AWS S3 or
> other object storage.  Those types of output storage are becoming more and
> more common as developers start distributing and testing builds across
> cloud environments.
>
> ​So please take a look at the proposal and give me any feedback or ideas
> and let me know if this is something I should try to move forward on.
>
> Thanks!
> Mark
>
> P.S.  I tested this on top of 2.5.1 and am running regression tests now on
> tip.
>
> --
> Mark Terrel  |  *Unbounded Systems*
>



-- 
Mark Terrel  |  *Unbounded Systems*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20170111/4743dc1f/attachment.html>