[Scons-users] Surprising behavior with pipes

[Jack Brennen]

This seems like more of a shell issue than a scons issue, but anywhere, 
here goes...

You could try something like this:

X=$(mktemp); (pipestep1 || rm -f $X) | (pipestep2 || rm -f $X) | (pipestep3 && rm $X)

It creates a temporary file.  If any of the early pipeline stages fails, 
delete the temporary file.
The final pipeline stage deletes the temp file, but if it was already 
deleted, the "rm" command will fail and trigger an error.
This should work with the Bourne shell or bash, or maybe others as long 
as they're Bourne-like, and not derived from csh or something like that.

(Note that it's not completely foolproof; it does depend on the idea 
that all of the commands which take a pipe as input read their entire 
input before returning SUCCESS.  But that's often a safe assumption.)

-Jack


On 10/1/2017 8:46 AM, Andrew C. Morrow wrote:
>
> That does assume that SHELL is set to bash though, right? I'd be OK 
> limiting it to POSIX, but going so far as to mandate one particular 
> shell implementation feels pretty fragile.
>
> I guess the right solution is to not use pipes, but instead redirect 
> the output to an intermediate target and then consume that, making a 
> two step process.
>
> Somewhat unfortunate in my case as I really don't need the 
> intermediate state and would prefer to avoid the IO cost. Perhaps the 
> intermediate could be a tempfile of some sort. Basically, re-implement 
> the pipe by hand.
>
>
> On Wed, Sep 20, 2017 at 12:51 PM, Gary Granger <granger at ucar.edu 
> <mailto:granger at ucar.edu>> wrote:
>
>     When I have had to use pipelines in actions, I have had to set
>     pipefail
>     explicitly, eg:
>
>     "set -o pipefail; %s 2>&1 | ${ASAN_FILTER}" % (command)
>
>
>     On 09/20/2017 08:39 AM, Andrew C. Morrow wrote:
>     >
>     > I noticed to my surprise recently that an Action that uses a
>     pipeline
>     > doesn't fail if one of the subcommands fails:
>     >
>     >
>     https://github.com/mongodb/mongo/blob/master/site_scons/site_tools/abilink.py#L66
>     <https://github.com/mongodb/mongo/blob/master/site_scons/site_tools/abilink.py#L66>
>     >
>     > I had a buggy version of abidw installed, and when it crashed
>     with an
>     > assertion, the SCons build did not terminate with an error. Instead
>     > the partial output up to the crash was consumed by md5 and used
>     as the
>     > result.
>     >
>     > Is there a way to ask SCons to enforce pipefail?
>     >
>
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20171002/4049d196/attachment.html>