[Scons-users] SCons misses a dependency which results in incorrect output.

Tal Dayan tal at zapta.com
Wed Jul 16 15:56:20 EDT 2025 

Hi Bill, I will file an issue.

I don't think it's safe for SCons to assume that once a program writes an
output file it can't crash.

On Wed, Jul 16, 2025 at 12:45 PM Bill Deegan <bill at baddogconsulting.com>
wrote:

> Arguably your command.sh is incorrectly written.
>
>  It copies the file and then checks it's contents and exits with error
> status if there improper.
>
> So a quick fix would be to check the input before copying the file.
>
> From the URL you posted in other email here's the command.sh
> #!/bin/bash
> # Usage: ./command input > output
>
> # Copy source to destination
> cp "$1" "$2"
>
> # If the input file stats with 'bad' inject an error AFTER creating the
> output file.
>
> first_line=$(head -n 1 "$1")
>
> if [[ "$first_line" == bad* ]]; then
>   exit 1
> fi
>
> exit 0
>
> Change it to this:
> #!/bin/bash
> # Usage: ./command input > output
>
> # If the input file stats with 'bad' inject an error AFTER creating the
> output file.
>
> first_line=$(head -n 1 "$1")
>
> if [[ "$first_line" == bad* ]]; then
>   exit 1
> fi
>
> # Copy source to destination
> cp "$1" "$2"
>
> exit 0
>
> Results:
>
> % ./run.sh
>
> ----- Iteration 1: file1 = 'good' -----
>
> scons: Reading SConscript files ...
> scons: done reading SConscript files.
> scons: Building targets ...
> scons: building `file2' because it doesn't exist
> ./command.sh file1 file2
> +-file2
>   +-file1
> scons: done building targets.
>
> File1
> MD5 (file1) = d7f986677d9f563bd1794b09d82206a3
>      1 good
>
> File2
> MD5 (file2) = d7f986677d9f563bd1794b09d82206a3
>      1 good
>
> DBlite:
> === .:
> file1: d7f986677d9f563bd1794b09d82206a3 1752694898 5
> file2: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>         file1: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>         2dbc2dce125a753309a27b7d5157aaaa [./command.sh $SOURCE $TARGET]
>
> ----- Iteration 2: file1 = 'bad' -----
>
> scons: Reading SConscript files ...
> scons: done reading SConscript files.
> scons: Building targets ...
> scons: rebuilding `file2' because `file1' changed
> ./command.sh file1 file2
> scons: *** [file2] Error 1
> +-file2
>   +-file1
> scons: building terminated because of errors.
>
> File1
> MD5 (file1) = df207dc9143c6fabf60b69b9c3035103
>      1 bad
>
> File2
> md5: file2: No such file or directory
> cat: file2: No such file or directory
>
> DBlite:
> === .:
> file1: df207dc9143c6fabf60b69b9c3035103 1752694898 4
> file2: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>         file1: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>         2dbc2dce125a753309a27b7d5157aaaa [./command.sh $SOURCE $TARGET]
>
> ----- Iteration 3: file1 = 'good' -----
>
> scons: Reading SConscript files ...
> scons: done reading SConscript files.
> scons: Building targets ...
> scons: building `file2' because it doesn't exist
> ./command.sh file1 file2
> +-file2
>   +-file1
> scons: done building targets.
>
> File1
> MD5 (file1) = d7f986677d9f563bd1794b09d82206a3
>      1 good
>
> File2
> MD5 (file2) = d7f986677d9f563bd1794b09d82206a3
>      1 good
>
> DBlite:
> === .:
> file1: d7f986677d9f563bd1794b09d82206a3 1752694899 5
> file2: d7f986677d9f563bd1794b09d82206a3 1752694899 5
>         file1: d7f986677d9f563bd1794b09d82206a3 1752694899 5
>         2dbc2dce125a753309a27b7d5157aaaa [./command.sh $SOURCE $TARGET]
>
> There's not presently logic in SCons to delete target files if the
> associated action yields an error.
>
> I think what you want is equivalent to makes .DELETE_ON_FAILURE, there's
> actually a SO question on this:
>
> https://stackoverflow.com/questions/29546276/scons-delete-target-on-failure-of-any-action
>
> Please go ahead and file an enhancement request to add equivalent
> to DELETE_ON_FAILURE, please include your reproducer scripts.
>
> -Bill
>
> On Wed, Jul 16, 2025 at 10:44 AM Tal Dayan <tal at zapta.com> wrote:
>
>> Looking at the end state of scons after invocation #3, the actual md5 of
>> file2 doesn't match its md5 in the dblite.
>>
>> https://i.imgur.com/NGco3yQ.png
>>
>> On Wed, Jul 16, 2025 at 10:33 AM Tal Dayan <tal at zapta.com> wrote:
>>
>>> Hi Keith, I updated the example files here
>>> https://github.com/FPGAwars/apio/issues/676
>>>
>>> They now include the md5 of the files and a dump of .sconsign.dblite
>>>
>>> On Wed, Jul 16, 2025 at 10:08 AM Keith Prussing <kprussing74 at gmail.com>
>>> wrote:
>>>
>>>> I suspect it's because `file1` has the same hash in the
>>>> .sconsign.dblite as the last "good" build (i.e. the first one). Thus
>>>> you get the line "scons: `file2' is up to date." on the third run.
>>>> However, I am not an expert in the specifics of SCons' hashing
>>>> methods.
>>>>
>>>> On Wed, Jul 16, 2025 at 12:38 PM Tal Dayan <tal at zapta.com> wrote:
>>>> >
>>>> > Hi all,
>>>> >
>>>> > We encountered this problem with the nextpnr tool and created here a
>>>> small and independent example that demonstrates it.
>>>> >
>>>> > In the example below, a shell script 'command.sh' reads the source
>>>> file 'file1' and writes it to the target file 'file2'. However, if the
>>>> > source file starts with 'bad' it exits with an error code, *after*
>>>> creating the target file.
>>>> >
>>>> > The script `run.sh', runs scons three times with these values of the
>>>> source file file1 'good', 'bad', and 'good'.  The expectation is that after
>>>> the third scons run, file2 should contain the value 'good' but it contains
>>>> the value 'bad'.
>>>> >
>>>> > Do we miss anything or is it simply a bug?
>>>> >
>>>> > SConstruct:
>>>> >
>>>> > ----------------------------------------------
>>>> >
>>>> > # SCons environment
>>>> >
>>>> > env = Environment()
>>>> >
>>>> >
>>>> > # Copy file1 → file2 using command.sh
>>>> >
>>>> > # Inject an error if file1 starts with 'bad"
>>>> >
>>>> > file2 = env.Command(
>>>> >
>>>> >     target='file2',
>>>> >
>>>> >     source='file1',
>>>> >
>>>> >     action='./command.sh $SOURCE > $TARGET'
>>>> >
>>>> > )
>>>> >
>>>> >
>>>> > # Make 'file2' the default target
>>>> >
>>>> > Default(file2)
>>>> >
>>>> > ----------------------------------------------
>>>> >
>>>> >
>>>> > command.sh:
>>>> >
>>>> > ----------------------------------------------
>>>> >
>>>> > #!/bin/bash
>>>> >
>>>> > # Usage: ./command input > output
>>>> >
>>>> >
>>>> > # Read from the first argument and copy to stdout
>>>> >
>>>> > cat "$1"
>>>> >
>>>> >
>>>> > # If the input file starts with 'bad', inject an error AFTER creating
>>>> the output file.
>>>> >
>>>> >
>>>> > first_line=$(head -n 1 "$1")
>>>> >
>>>> >
>>>> > if [[ "$first_line" == bad* ]]; then
>>>> >
>>>> >   exit 1
>>>> >
>>>> > fi
>>>> >
>>>> >
>>>> > exit 0
>>>> >
>>>> > ----------------------------------------------
>>>> >
>>>> >
>>>> >
>>>> > run.sh
>>>> >
>>>> > ----------------------------------------------
>>>> >
>>>> > #!/bin/bash
>>>> >
>>>> >
>>>> > # Clean up.
>>>> >
>>>> > rm -f .sconsign.dblite
>>>> >
>>>> > rm -f file[12]
>>>> >
>>>> >
>>>> > echo
>>>> >
>>>> > echo "---- Iteration 1: file1 = 'good'"
>>>> >
>>>> > echo "good" > file1
>>>> >
>>>> > scons
>>>> >
>>>> > echo
>>>> >
>>>> >
>>>> > echo "File1"
>>>> >
>>>> > cat -n file1
>>>> >
>>>> >
>>>> > echo "File2"
>>>> >
>>>> > cat -n file2
>>>> >
>>>> >
>>>> >
>>>> > echo
>>>> >
>>>> > echo "---- Iteration 2: file1 = 'bad'"
>>>> >
>>>> > echo "bad" > file1
>>>> >
>>>> > cat -n file1
>>>> >
>>>> > scons
>>>> >
>>>> > echo
>>>> >
>>>> >
>>>> > echo "File1"
>>>> >
>>>> > cat -n file1
>>>> >
>>>> >
>>>> > echo "File2"
>>>> >
>>>> > cat -n file2
>>>> >
>>>> >
>>>> > echo
>>>> >
>>>> > echo "---- Iteration 3: file1 = 'good'"
>>>> >
>>>> > echo "good" > file1
>>>> >
>>>> > cat -n file1
>>>> >
>>>> > scons
>>>> >
>>>> > echo
>>>> >
>>>> >
>>>> > echo "File1"
>>>> >
>>>> > cat -n file1
>>>> >
>>>> >
>>>> > echo "File2"
>>>> >
>>>> > cat -n file2
>>>> >
>>>> > ----------------------------------------------
>>>> >
>>>> >
>>>> >
>>>> > Run log:
>>>> >
>>>> > ----------------------------------------------
>>>> >
>>>> > $ ./run.sh
>>>> >
>>>> >
>>>> > ---- Iteration 1: file1 = 'good'
>>>> >
>>>> > scons: Reading SConscript files ...
>>>> >
>>>> > scons: done reading SConscript files.
>>>> >
>>>> > scons: Building targets ...
>>>> >
>>>> > ./command.sh file1 > file2
>>>> >
>>>> > scons: done building targets.
>>>> >
>>>> >
>>>> > File1
>>>> >
>>>> >      1 good
>>>> >
>>>> > File2
>>>> >
>>>> >      1 good
>>>> >
>>>> >
>>>> > ---- Iteration 2: file1 = 'bad'
>>>> >
>>>> >      1 bad
>>>> >
>>>> > scons: Reading SConscript files ...
>>>> >
>>>> > scons: done reading SConscript files.
>>>> >
>>>> > scons: Building targets ...
>>>> >
>>>> > ./command.sh file1 > file2
>>>> >
>>>> > scons: *** [file2] Error 1
>>>> >
>>>> > scons: building terminated because of errors.
>>>> >
>>>> >
>>>> > File1
>>>> >
>>>> >      1 bad
>>>> >
>>>> > File2
>>>> >
>>>> >      1 bad
>>>> >
>>>> >
>>>> > ---- Iteration 1: file1 = 'good'
>>>> >
>>>> >      1 good
>>>> >
>>>> > scons: Reading SConscript files ...
>>>> >
>>>> > scons: done reading SConscript files.
>>>> >
>>>> > scons: Building targets ...
>>>> >
>>>> > scons: `file2' is up to date.
>>>> >
>>>> > scons: done building targets.
>>>> >
>>>> >
>>>> > File1
>>>> >
>>>> >      1 good
>>>> >
>>>> > File2
>>>> >
>>>> >      1 bad
>>>> >
>>>> > ----------------------------------------------
>>>> >
>>>> > _______________________________________________
>>>> > Scons-users mailing list
>>>> > Scons-users at scons.org
>>>> > https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>>>
>>>>
>>>> --
>>>> Keith Prussing
>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20250716/033f9061/attachment-0001.htm>

More information about the Scons-users mailing list