[Scons-users] SCons misses a dependency which results in incorrect output.

Bill Deegan bill at baddogconsulting.com
Wed Jul 16 18:16:51 EDT 2025 

It's an enhancement request.

The bug is in your command.sh script.
I'll change the description and text.



On Wed, Jul 16, 2025 at 12:56 PM Tal Dayan <tal at zapta.com> wrote:

> Hi Bill, I will file an issue.
>
> I don't think it's safe for SCons to assume that once a program writes an
> output file it can't crash.
>
> On Wed, Jul 16, 2025 at 12:45 PM Bill Deegan <bill at baddogconsulting.com>
> wrote:
>
>> Arguably your command.sh is incorrectly written.
>>
>>  It copies the file and then checks it's contents and exits with error
>> status if there improper.
>>
>> So a quick fix would be to check the input before copying the file.
>>
>> From the URL you posted in other email here's the command.sh
>> #!/bin/bash
>> # Usage: ./command input > output
>>
>> # Copy source to destination
>> cp "$1" "$2"
>>
>> # If the input file stats with 'bad' inject an error AFTER creating the
>> output file.
>>
>> first_line=$(head -n 1 "$1")
>>
>> if [[ "$first_line" == bad* ]]; then
>>   exit 1
>> fi
>>
>> exit 0
>>
>> Change it to this:
>> #!/bin/bash
>> # Usage: ./command input > output
>>
>> # If the input file stats with 'bad' inject an error AFTER creating the
>> output file.
>>
>> first_line=$(head -n 1 "$1")
>>
>> if [[ "$first_line" == bad* ]]; then
>>   exit 1
>> fi
>>
>> # Copy source to destination
>> cp "$1" "$2"
>>
>> exit 0
>>
>> Results:
>>
>> % ./run.sh
>>
>> ----- Iteration 1: file1 = 'good' -----
>>
>> scons: Reading SConscript files ...
>> scons: done reading SConscript files.
>> scons: Building targets ...
>> scons: building `file2' because it doesn't exist
>> ./command.sh file1 file2
>> +-file2
>>   +-file1
>> scons: done building targets.
>>
>> File1
>> MD5 (file1) = d7f986677d9f563bd1794b09d82206a3
>>      1 good
>>
>> File2
>> MD5 (file2) = d7f986677d9f563bd1794b09d82206a3
>>      1 good
>>
>> DBlite:
>> === .:
>> file1: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>> file2: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>         file1: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>         2dbc2dce125a753309a27b7d5157aaaa [./command.sh $SOURCE $TARGET]
>>
>> ----- Iteration 2: file1 = 'bad' -----
>>
>> scons: Reading SConscript files ...
>> scons: done reading SConscript files.
>> scons: Building targets ...
>> scons: rebuilding `file2' because `file1' changed
>> ./command.sh file1 file2
>> scons: *** [file2] Error 1
>> +-file2
>>   +-file1
>> scons: building terminated because of errors.
>>
>> File1
>> MD5 (file1) = df207dc9143c6fabf60b69b9c3035103
>>      1 bad
>>
>> File2
>> md5: file2: No such file or directory
>> cat: file2: No such file or directory
>>
>> DBlite:
>> === .:
>> file1: df207dc9143c6fabf60b69b9c3035103 1752694898 4
>> file2: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>         file1: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>         2dbc2dce125a753309a27b7d5157aaaa [./command.sh $SOURCE $TARGET]
>>
>> ----- Iteration 3: file1 = 'good' -----
>>
>> scons: Reading SConscript files ...
>> scons: done reading SConscript files.
>> scons: Building targets ...
>> scons: building `file2' because it doesn't exist
>> ./command.sh file1 file2
>> +-file2
>>   +-file1
>> scons: done building targets.
>>
>> File1
>> MD5 (file1) = d7f986677d9f563bd1794b09d82206a3
>>      1 good
>>
>> File2
>> MD5 (file2) = d7f986677d9f563bd1794b09d82206a3
>>      1 good
>>
>> DBlite:
>> === .:
>> file1: d7f986677d9f563bd1794b09d82206a3 1752694899 5
>> file2: d7f986677d9f563bd1794b09d82206a3 1752694899 5
>>         file1: d7f986677d9f563bd1794b09d82206a3 1752694899 5
>>         2dbc2dce125a753309a27b7d5157aaaa [./command.sh $SOURCE $TARGET]
>>
>> There's not presently logic in SCons to delete target files if the
>> associated action yields an error.
>>
>> I think what you want is equivalent to makes .DELETE_ON_FAILURE, there's
>> actually a SO question on this:
>>
>> https://stackoverflow.com/questions/29546276/scons-delete-target-on-failure-of-any-action
>>
>> Please go ahead and file an enhancement request to add equivalent
>> to DELETE_ON_FAILURE, please include your reproducer scripts.
>>
>> -Bill
>>
>> On Wed, Jul 16, 2025 at 10:44 AM Tal Dayan <tal at zapta.com> wrote:
>>
>>> Looking at the end state of scons after invocation #3, the actual md5 of
>>> file2 doesn't match its md5 in the dblite.
>>>
>>> https://i.imgur.com/NGco3yQ.png
>>>
>>> On Wed, Jul 16, 2025 at 10:33 AM Tal Dayan <tal at zapta.com> wrote:
>>>
>>>> Hi Keith, I updated the example files here
>>>> https://github.com/FPGAwars/apio/issues/676
>>>>
>>>> They now include the md5 of the files and a dump of .sconsign.dblite
>>>>
>>>> On Wed, Jul 16, 2025 at 10:08 AM Keith Prussing <kprussing74 at gmail.com>
>>>> wrote:
>>>>
>>>>> I suspect it's because `file1` has the same hash in the
>>>>> .sconsign.dblite as the last "good" build (i.e. the first one). Thus
>>>>> you get the line "scons: `file2' is up to date." on the third run.
>>>>> However, I am not an expert in the specifics of SCons' hashing
>>>>> methods.
>>>>>
>>>>> On Wed, Jul 16, 2025 at 12:38 PM Tal Dayan <tal at zapta.com> wrote:
>>>>> >
>>>>> > Hi all,
>>>>> >
>>>>> > We encountered this problem with the nextpnr tool and created here a
>>>>> small and independent example that demonstrates it.
>>>>> >
>>>>> > In the example below, a shell script 'command.sh' reads the source
>>>>> file 'file1' and writes it to the target file 'file2'. However, if the
>>>>> > source file starts with 'bad' it exits with an error code, *after*
>>>>> creating the target file.
>>>>> >
>>>>> > The script `run.sh', runs scons three times with these values of the
>>>>> source file file1 'good', 'bad', and 'good'.  The expectation is that after
>>>>> the third scons run, file2 should contain the value 'good' but it contains
>>>>> the value 'bad'.
>>>>> >
>>>>> > Do we miss anything or is it simply a bug?
>>>>> >
>>>>> > SConstruct:
>>>>> >
>>>>> > ----------------------------------------------
>>>>> >
>>>>> > # SCons environment
>>>>> >
>>>>> > env = Environment()
>>>>> >
>>>>> >
>>>>> > # Copy file1 → file2 using command.sh
>>>>> >
>>>>> > # Inject an error if file1 starts with 'bad"
>>>>> >
>>>>> > file2 = env.Command(
>>>>> >
>>>>> >     target='file2',
>>>>> >
>>>>> >     source='file1',
>>>>> >
>>>>> >     action='./command.sh $SOURCE > $TARGET'
>>>>> >
>>>>> > )
>>>>> >
>>>>> >
>>>>> > # Make 'file2' the default target
>>>>> >
>>>>> > Default(file2)
>>>>> >
>>>>> > ----------------------------------------------
>>>>> >
>>>>> >
>>>>> > command.sh:
>>>>> >
>>>>> > ----------------------------------------------
>>>>> >
>>>>> > #!/bin/bash
>>>>> >
>>>>> > # Usage: ./command input > output
>>>>> >
>>>>> >
>>>>> > # Read from the first argument and copy to stdout
>>>>> >
>>>>> > cat "$1"
>>>>> >
>>>>> >
>>>>> > # If the input file starts with 'bad', inject an error AFTER
>>>>> creating the output file.
>>>>> >
>>>>> >
>>>>> > first_line=$(head -n 1 "$1")
>>>>> >
>>>>> >
>>>>> > if [[ "$first_line" == bad* ]]; then
>>>>> >
>>>>> >   exit 1
>>>>> >
>>>>> > fi
>>>>> >
>>>>> >
>>>>> > exit 0
>>>>> >
>>>>> > ----------------------------------------------
>>>>> >
>>>>> >
>>>>> >
>>>>> > run.sh
>>>>> >
>>>>> > ----------------------------------------------
>>>>> >
>>>>> > #!/bin/bash
>>>>> >
>>>>> >
>>>>> > # Clean up.
>>>>> >
>>>>> > rm -f .sconsign.dblite
>>>>> >
>>>>> > rm -f file[12]
>>>>> >
>>>>> >
>>>>> > echo
>>>>> >
>>>>> > echo "---- Iteration 1: file1 = 'good'"
>>>>> >
>>>>> > echo "good" > file1
>>>>> >
>>>>> > scons
>>>>> >
>>>>> > echo
>>>>> >
>>>>> >
>>>>> > echo "File1"
>>>>> >
>>>>> > cat -n file1
>>>>> >
>>>>> >
>>>>> > echo "File2"
>>>>> >
>>>>> > cat -n file2
>>>>> >
>>>>> >
>>>>> >
>>>>> > echo
>>>>> >
>>>>> > echo "---- Iteration 2: file1 = 'bad'"
>>>>> >
>>>>> > echo "bad" > file1
>>>>> >
>>>>> > cat -n file1
>>>>> >
>>>>> > scons
>>>>> >
>>>>> > echo
>>>>> >
>>>>> >
>>>>> > echo "File1"
>>>>> >
>>>>> > cat -n file1
>>>>> >
>>>>> >
>>>>> > echo "File2"
>>>>> >
>>>>> > cat -n file2
>>>>> >
>>>>> >
>>>>> > echo
>>>>> >
>>>>> > echo "---- Iteration 3: file1 = 'good'"
>>>>> >
>>>>> > echo "good" > file1
>>>>> >
>>>>> > cat -n file1
>>>>> >
>>>>> > scons
>>>>> >
>>>>> > echo
>>>>> >
>>>>> >
>>>>> > echo "File1"
>>>>> >
>>>>> > cat -n file1
>>>>> >
>>>>> >
>>>>> > echo "File2"
>>>>> >
>>>>> > cat -n file2
>>>>> >
>>>>> > ----------------------------------------------
>>>>> >
>>>>> >
>>>>> >
>>>>> > Run log:
>>>>> >
>>>>> > ----------------------------------------------
>>>>> >
>>>>> > $ ./run.sh
>>>>> >
>>>>> >
>>>>> > ---- Iteration 1: file1 = 'good'
>>>>> >
>>>>> > scons: Reading SConscript files ...
>>>>> >
>>>>> > scons: done reading SConscript files.
>>>>> >
>>>>> > scons: Building targets ...
>>>>> >
>>>>> > ./command.sh file1 > file2
>>>>> >
>>>>> > scons: done building targets.
>>>>> >
>>>>> >
>>>>> > File1
>>>>> >
>>>>> >      1 good
>>>>> >
>>>>> > File2
>>>>> >
>>>>> >      1 good
>>>>> >
>>>>> >
>>>>> > ---- Iteration 2: file1 = 'bad'
>>>>> >
>>>>> >      1 bad
>>>>> >
>>>>> > scons: Reading SConscript files ...
>>>>> >
>>>>> > scons: done reading SConscript files.
>>>>> >
>>>>> > scons: Building targets ...
>>>>> >
>>>>> > ./command.sh file1 > file2
>>>>> >
>>>>> > scons: *** [file2] Error 1
>>>>> >
>>>>> > scons: building terminated because of errors.
>>>>> >
>>>>> >
>>>>> > File1
>>>>> >
>>>>> >      1 bad
>>>>> >
>>>>> > File2
>>>>> >
>>>>> >      1 bad
>>>>> >
>>>>> >
>>>>> > ---- Iteration 1: file1 = 'good'
>>>>> >
>>>>> >      1 good
>>>>> >
>>>>> > scons: Reading SConscript files ...
>>>>> >
>>>>> > scons: done reading SConscript files.
>>>>> >
>>>>> > scons: Building targets ...
>>>>> >
>>>>> > scons: `file2' is up to date.
>>>>> >
>>>>> > scons: done building targets.
>>>>> >
>>>>> >
>>>>> > File1
>>>>> >
>>>>> >      1 good
>>>>> >
>>>>> > File2
>>>>> >
>>>>> >      1 bad
>>>>> >
>>>>> > ----------------------------------------------
>>>>> >
>>>>> > _______________________________________________
>>>>> > Scons-users mailing list
>>>>> > Scons-users at scons.org
>>>>> > https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Keith Prussing
>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20250716/164bdd0e/attachment-0001.htm>

More information about the Scons-users mailing list