[Scons-users] Detecting changed targets

[William Blevins]

Mark,

I am on board with a change like this. It has been proposed or discussed
several times. I dabbled with it a while back to see if it was feasible:
https://bitbucket.org/scons/scons/pull-requests/240/added-support-for-detecting-corruption/diff

Problem was that this broke tons of tests and I didn't get much in the way
of enthusiasm at the time. How our the tests under your patch? You can run
them like "./runtests.py -a -j <cpus>" from the repo.

V/R,
William

On Wed, Jan 11, 2017 at 12:10 PM, Mark Terrel <mark at unbounded.systems>
wrote:

> Hi SCons folks,
>
> Now that the holidays are over, I was hoping someone might give me
> feedback on my proposal.  Any thoughts?
>
> Thanks,
> Mark
>
> On Wed, Dec 28, 2016 at 3:01 PM, Mark Terrel <mark at unbounded.systems>
> wrote:
>
>> Hi SCons gurus!
>>
>> I've been a user of SCons since 2010 (at a previous company) and am using
>> it for a new project at a new company now.  We're doing all builds and
>> development in containers, so I've created a SCons builder to build Docker
>> images.  (I'm happy to share that once it's had more testing.)  That's
>> working great so far, except for one issue that I'd like to solve.
>>
>> The issue stems from the fact that Docker images are not stored in the
>> filesystem; they're stored in a common object store repository and are
>> tagged with a name.  So while a typical user of SCons has a reasonable
>> expectation that no one is going to come modify their build output
>> directory contents, it now becomes much more likely that Developer A builds
>> a Docker image with tag "production_container" and Developer B could build
>> a different image and overwrite (actually move the tag)
>> "production_container" to point to a different Docker image.  This makes it
>> harder to ensure correctness of builds because SCons does not currently
>> rebuild a target if the signature of the target changes.
>>
>> I found this exchange
>> <http://scons-users.scons.narkive.com/7BJnwEox/corrupted-objects-are-not-rebuilt-as-one-might-expect>
>> on the scons-users list from ages ago, where Steven Knight suggested a new
>> ValidateTargets option that would detect if a target was changed (or had
>> been corrupted) unbeknownst to SCons and would then rebuild that target.
>> So I took a shot at implementing that feature and would love some feedback
>> on whether this is a good approach and whether, after it meets the dev
>> team's approval, you might be willing to incorporate it.
>>
>> This commit
>> <https://bitbucket.org/mterrel/scons/commits/1b0c0f22d13596f709b6e3ca64a8f2ef41fd4c11>contains
>> the proposed feature as a proof of concept.  It has two new user-facing
>> functions:
>>
>> *ValidateTargets('none' | 'content')*
>>
>> Sets the global default behavior for all target nodes.
>> 'none' is the default and is the same as SCons' current behavior, which
>> is to not check the signatures for nodes that are only targets.
>>
>> 'content' uses the existing content signatures to check whether a target
>> node has been modified outside of SCons' knowledge.
>>
>> *env.ValidateTarget(target_list, 'none' | 'content')*
>>
>> Sets the validation behavior for specific node(s), overriding the global
>> default behavior for those nodes.  Options 'none' and 'content' behave the
>> same as above.
>>
>>
>> ​I think this new behavior contributes toward one of SCons' goals of
>> correctness of builds.  I think it would also be particularly useful for
>> other kinds of targets that might be in shared storage, such as AWS S3 or
>> other object storage.  Those types of output storage are becoming more and
>> more common as developers start distributing and testing builds across
>> cloud environments.
>>
>> ​So please take a look at the proposal and give me any feedback or ideas
>> and let me know if this is something I should try to move forward on.
>>
>> Thanks!
>> Mark
>>
>> P.S.  I tested this on top of 2.5.1 and am running regression tests now
>> on tip.
>>
>> --
>> Mark Terrel  |  *Unbounded Systems*
>>
>
>
>
> --
> Mark Terrel  |  *Unbounded Systems*
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20170111/c07de779/attachment-0001.html>