[Scons-users] SCons misses a dependency which results in incorrect output.

Bill Deegan bill at baddogconsulting.com
Wed Jul 16 18:18:54 EDT 2025 

Did you read my explanation and how to fix it in your environment assuming
command.sh is similar to what's really happening in your build?


On Wed, Jul 16, 2025 at 3:16 PM Bill Deegan <bill at baddogconsulting.com>
wrote:

> It's an enhancement request.
>
> The bug is in your command.sh script.
> I'll change the description and text.
>
>
>
> On Wed, Jul 16, 2025 at 12:56 PM Tal Dayan <tal at zapta.com> wrote:
>
>> Hi Bill, I will file an issue.
>>
>> I don't think it's safe for SCons to assume that once a program writes an
>> output file it can't crash.
>>
>> On Wed, Jul 16, 2025 at 12:45 PM Bill Deegan <bill at baddogconsulting.com>
>> wrote:
>>
>>> Arguably your command.sh is incorrectly written.
>>>
>>>  It copies the file and then checks it's contents and exits with error
>>> status if there improper.
>>>
>>> So a quick fix would be to check the input before copying the file.
>>>
>>> From the URL you posted in other email here's the command.sh
>>> #!/bin/bash
>>> # Usage: ./command input > output
>>>
>>> # Copy source to destination
>>> cp "$1" "$2"
>>>
>>> # If the input file stats with 'bad' inject an error AFTER creating the
>>> output file.
>>>
>>> first_line=$(head -n 1 "$1")
>>>
>>> if [[ "$first_line" == bad* ]]; then
>>>   exit 1
>>> fi
>>>
>>> exit 0
>>>
>>> Change it to this:
>>> #!/bin/bash
>>> # Usage: ./command input > output
>>>
>>> # If the input file stats with 'bad' inject an error AFTER creating the
>>> output file.
>>>
>>> first_line=$(head -n 1 "$1")
>>>
>>> if [[ "$first_line" == bad* ]]; then
>>>   exit 1
>>> fi
>>>
>>> # Copy source to destination
>>> cp "$1" "$2"
>>>
>>> exit 0
>>>
>>> Results:
>>>
>>> % ./run.sh
>>>
>>> ----- Iteration 1: file1 = 'good' -----
>>>
>>> scons: Reading SConscript files ...
>>> scons: done reading SConscript files.
>>> scons: Building targets ...
>>> scons: building `file2' because it doesn't exist
>>> ./command.sh file1 file2
>>> +-file2
>>>   +-file1
>>> scons: done building targets.
>>>
>>> File1
>>> MD5 (file1) = d7f986677d9f563bd1794b09d82206a3
>>>      1 good
>>>
>>> File2
>>> MD5 (file2) = d7f986677d9f563bd1794b09d82206a3
>>>      1 good
>>>
>>> DBlite:
>>> === .:
>>> file1: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>> file2: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>>         file1: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>>         2dbc2dce125a753309a27b7d5157aaaa [./command.sh $SOURCE $TARGET]
>>>
>>> ----- Iteration 2: file1 = 'bad' -----
>>>
>>> scons: Reading SConscript files ...
>>> scons: done reading SConscript files.
>>> scons: Building targets ...
>>> scons: rebuilding `file2' because `file1' changed
>>> ./command.sh file1 file2
>>> scons: *** [file2] Error 1
>>> +-file2
>>>   +-file1
>>> scons: building terminated because of errors.
>>>
>>> File1
>>> MD5 (file1) = df207dc9143c6fabf60b69b9c3035103
>>>      1 bad
>>>
>>> File2
>>> md5: file2: No such file or directory
>>> cat: file2: No such file or directory
>>>
>>> DBlite:
>>> === .:
>>> file1: df207dc9143c6fabf60b69b9c3035103 1752694898 4
>>> file2: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>>         file1: d7f986677d9f563bd1794b09d82206a3 1752694898 5
>>>         2dbc2dce125a753309a27b7d5157aaaa [./command.sh $SOURCE $TARGET]
>>>
>>> ----- Iteration 3: file1 = 'good' -----
>>>
>>> scons: Reading SConscript files ...
>>> scons: done reading SConscript files.
>>> scons: Building targets ...
>>> scons: building `file2' because it doesn't exist
>>> ./command.sh file1 file2
>>> +-file2
>>>   +-file1
>>> scons: done building targets.
>>>
>>> File1
>>> MD5 (file1) = d7f986677d9f563bd1794b09d82206a3
>>>      1 good
>>>
>>> File2
>>> MD5 (file2) = d7f986677d9f563bd1794b09d82206a3
>>>      1 good
>>>
>>> DBlite:
>>> === .:
>>> file1: d7f986677d9f563bd1794b09d82206a3 1752694899 5
>>> file2: d7f986677d9f563bd1794b09d82206a3 1752694899 5
>>>         file1: d7f986677d9f563bd1794b09d82206a3 1752694899 5
>>>         2dbc2dce125a753309a27b7d5157aaaa [./command.sh $SOURCE $TARGET]
>>>
>>> There's not presently logic in SCons to delete target files if the
>>> associated action yields an error.
>>>
>>> I think what you want is equivalent to makes .DELETE_ON_FAILURE, there's
>>> actually a SO question on this:
>>>
>>> https://stackoverflow.com/questions/29546276/scons-delete-target-on-failure-of-any-action
>>>
>>> Please go ahead and file an enhancement request to add equivalent
>>> to DELETE_ON_FAILURE, please include your reproducer scripts.
>>>
>>> -Bill
>>>
>>> On Wed, Jul 16, 2025 at 10:44 AM Tal Dayan <tal at zapta.com> wrote:
>>>
>>>> Looking at the end state of scons after invocation #3, the actual md5
>>>> of file2 doesn't match its md5 in the dblite.
>>>>
>>>> https://i.imgur.com/NGco3yQ.png
>>>>
>>>> On Wed, Jul 16, 2025 at 10:33 AM Tal Dayan <tal at zapta.com> wrote:
>>>>
>>>>> Hi Keith, I updated the example files here
>>>>> https://github.com/FPGAwars/apio/issues/676
>>>>>
>>>>> They now include the md5 of the files and a dump of .sconsign.dblite
>>>>>
>>>>> On Wed, Jul 16, 2025 at 10:08 AM Keith Prussing <kprussing74 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I suspect it's because `file1` has the same hash in the
>>>>>> .sconsign.dblite as the last "good" build (i.e. the first one). Thus
>>>>>> you get the line "scons: `file2' is up to date." on the third run.
>>>>>> However, I am not an expert in the specifics of SCons' hashing
>>>>>> methods.
>>>>>>
>>>>>> On Wed, Jul 16, 2025 at 12:38 PM Tal Dayan <tal at zapta.com> wrote:
>>>>>> >
>>>>>> > Hi all,
>>>>>> >
>>>>>> > We encountered this problem with the nextpnr tool and created here
>>>>>> a small and independent example that demonstrates it.
>>>>>> >
>>>>>> > In the example below, a shell script 'command.sh' reads the source
>>>>>> file 'file1' and writes it to the target file 'file2'. However, if the
>>>>>> > source file starts with 'bad' it exits with an error code, *after*
>>>>>> creating the target file.
>>>>>> >
>>>>>> > The script `run.sh', runs scons three times with these values of
>>>>>> the source file file1 'good', 'bad', and 'good'.  The expectation is that
>>>>>> after the third scons run, file2 should contain the value 'good' but it
>>>>>> contains the value 'bad'.
>>>>>> >
>>>>>> > Do we miss anything or is it simply a bug?
>>>>>> >
>>>>>> > SConstruct:
>>>>>> >
>>>>>> > ----------------------------------------------
>>>>>> >
>>>>>> > # SCons environment
>>>>>> >
>>>>>> > env = Environment()
>>>>>> >
>>>>>> >
>>>>>> > # Copy file1 → file2 using command.sh
>>>>>> >
>>>>>> > # Inject an error if file1 starts with 'bad"
>>>>>> >
>>>>>> > file2 = env.Command(
>>>>>> >
>>>>>> >     target='file2',
>>>>>> >
>>>>>> >     source='file1',
>>>>>> >
>>>>>> >     action='./command.sh $SOURCE > $TARGET'
>>>>>> >
>>>>>> > )
>>>>>> >
>>>>>> >
>>>>>> > # Make 'file2' the default target
>>>>>> >
>>>>>> > Default(file2)
>>>>>> >
>>>>>> > ----------------------------------------------
>>>>>> >
>>>>>> >
>>>>>> > command.sh:
>>>>>> >
>>>>>> > ----------------------------------------------
>>>>>> >
>>>>>> > #!/bin/bash
>>>>>> >
>>>>>> > # Usage: ./command input > output
>>>>>> >
>>>>>> >
>>>>>> > # Read from the first argument and copy to stdout
>>>>>> >
>>>>>> > cat "$1"
>>>>>> >
>>>>>> >
>>>>>> > # If the input file starts with 'bad', inject an error AFTER
>>>>>> creating the output file.
>>>>>> >
>>>>>> >
>>>>>> > first_line=$(head -n 1 "$1")
>>>>>> >
>>>>>> >
>>>>>> > if [[ "$first_line" == bad* ]]; then
>>>>>> >
>>>>>> >   exit 1
>>>>>> >
>>>>>> > fi
>>>>>> >
>>>>>> >
>>>>>> > exit 0
>>>>>> >
>>>>>> > ----------------------------------------------
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > run.sh
>>>>>> >
>>>>>> > ----------------------------------------------
>>>>>> >
>>>>>> > #!/bin/bash
>>>>>> >
>>>>>> >
>>>>>> > # Clean up.
>>>>>> >
>>>>>> > rm -f .sconsign.dblite
>>>>>> >
>>>>>> > rm -f file[12]
>>>>>> >
>>>>>> >
>>>>>> > echo
>>>>>> >
>>>>>> > echo "---- Iteration 1: file1 = 'good'"
>>>>>> >
>>>>>> > echo "good" > file1
>>>>>> >
>>>>>> > scons
>>>>>> >
>>>>>> > echo
>>>>>> >
>>>>>> >
>>>>>> > echo "File1"
>>>>>> >
>>>>>> > cat -n file1
>>>>>> >
>>>>>> >
>>>>>> > echo "File2"
>>>>>> >
>>>>>> > cat -n file2
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > echo
>>>>>> >
>>>>>> > echo "---- Iteration 2: file1 = 'bad'"
>>>>>> >
>>>>>> > echo "bad" > file1
>>>>>> >
>>>>>> > cat -n file1
>>>>>> >
>>>>>> > scons
>>>>>> >
>>>>>> > echo
>>>>>> >
>>>>>> >
>>>>>> > echo "File1"
>>>>>> >
>>>>>> > cat -n file1
>>>>>> >
>>>>>> >
>>>>>> > echo "File2"
>>>>>> >
>>>>>> > cat -n file2
>>>>>> >
>>>>>> >
>>>>>> > echo
>>>>>> >
>>>>>> > echo "---- Iteration 3: file1 = 'good'"
>>>>>> >
>>>>>> > echo "good" > file1
>>>>>> >
>>>>>> > cat -n file1
>>>>>> >
>>>>>> > scons
>>>>>> >
>>>>>> > echo
>>>>>> >
>>>>>> >
>>>>>> > echo "File1"
>>>>>> >
>>>>>> > cat -n file1
>>>>>> >
>>>>>> >
>>>>>> > echo "File2"
>>>>>> >
>>>>>> > cat -n file2
>>>>>> >
>>>>>> > ----------------------------------------------
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > Run log:
>>>>>> >
>>>>>> > ----------------------------------------------
>>>>>> >
>>>>>> > $ ./run.sh
>>>>>> >
>>>>>> >
>>>>>> > ---- Iteration 1: file1 = 'good'
>>>>>> >
>>>>>> > scons: Reading SConscript files ...
>>>>>> >
>>>>>> > scons: done reading SConscript files.
>>>>>> >
>>>>>> > scons: Building targets ...
>>>>>> >
>>>>>> > ./command.sh file1 > file2
>>>>>> >
>>>>>> > scons: done building targets.
>>>>>> >
>>>>>> >
>>>>>> > File1
>>>>>> >
>>>>>> >      1 good
>>>>>> >
>>>>>> > File2
>>>>>> >
>>>>>> >      1 good
>>>>>> >
>>>>>> >
>>>>>> > ---- Iteration 2: file1 = 'bad'
>>>>>> >
>>>>>> >      1 bad
>>>>>> >
>>>>>> > scons: Reading SConscript files ...
>>>>>> >
>>>>>> > scons: done reading SConscript files.
>>>>>> >
>>>>>> > scons: Building targets ...
>>>>>> >
>>>>>> > ./command.sh file1 > file2
>>>>>> >
>>>>>> > scons: *** [file2] Error 1
>>>>>> >
>>>>>> > scons: building terminated because of errors.
>>>>>> >
>>>>>> >
>>>>>> > File1
>>>>>> >
>>>>>> >      1 bad
>>>>>> >
>>>>>> > File2
>>>>>> >
>>>>>> >      1 bad
>>>>>> >
>>>>>> >
>>>>>> > ---- Iteration 1: file1 = 'good'
>>>>>> >
>>>>>> >      1 good
>>>>>> >
>>>>>> > scons: Reading SConscript files ...
>>>>>> >
>>>>>> > scons: done reading SConscript files.
>>>>>> >
>>>>>> > scons: Building targets ...
>>>>>> >
>>>>>> > scons: `file2' is up to date.
>>>>>> >
>>>>>> > scons: done building targets.
>>>>>> >
>>>>>> >
>>>>>> > File1
>>>>>> >
>>>>>> >      1 good
>>>>>> >
>>>>>> > File2
>>>>>> >
>>>>>> >      1 bad
>>>>>> >
>>>>>> > ----------------------------------------------
>>>>>> >
>>>>>> > _______________________________________________
>>>>>> > Scons-users mailing list
>>>>>> > Scons-users at scons.org
>>>>>> > https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Keith Prussing
>>>>>> _______________________________________________
>>>>>> Scons-users mailing list
>>>>>> Scons-users at scons.org
>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>
>>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20250716/2c05359b/attachment-0001.htm>

More information about the Scons-users mailing list